FP7 Space ResearchProposal evaluationand role of the REAChristine BernotHead of UnitEuropean CommissionREA S2 Space ResearchRome,20/09/2012
Overview1. REA missionREA activities & Space ResearchRole of REA & ECGMES - SSF - Cross-cutting budget2. Understanding the evaluation processEvaluation roadmap for the callFP7-SPACE-2013-1Evaluation processESA input3. Key highlights on proposal preparationEligibilityEvaluation criteriaAdditional explanations and hintsWhere to find relevant information
Space Researchat the REAThe REA Space Research unit is responsible forimplementing the part of the Space work programmededicated to co-financing research projectsIn practice it means:•Organising and being responsible for the evaluation•Negotiating projects, preparing and signing financial and legalcommitments•Monitoring projects progress, dealing with contractual issues•Checking on the use of resources and making payments•Implementing audit results
REA & ECThe European Commission’s role inFP7 Space research•Policy work remains within theCommission - DG Enterprise (ENTR)> this includes the definition of theSpace work programme•DG ENTR finances directly specifictopics identified in the workprogramme(e.g. development of the Spacecomponent of GMES)•Space Research budget: 1.43 billionaround 640 externalised to REA
InteractionREA.S2 - ENTR.H2• Smooth cooperation on evaluation, negotiation, projectimplementation, dissemination...DG ENTR• Work programme definition• Dealing with ProgrammeCommittee (PC)• Presentation of the WP andsupport to applicants• Presentation to PC• Inter-Services Consultation• Commission selection Decision• Development of WP, publicationon projects, organisation ofconferencesREA• Guides for applicants, Callpublication• Organising the evaluation• Support to applicants• Evaluation of proposals• Reporting on evaluation outcome• Negotiations process• End of negotiations• Financial commitment andsignature of grant agreement• Information on projects, successstories, contributions toconferences & publications
EU funding peractivity (and call)CC 7%SSF 38%€120.000.000M€141,9€100.000.000GMES 55%€80.000.000€60.000.000M€121,8M€107,3GMESSSFM€ 285,6M€ 193,7€40.000.000M€54,6M€89,1CCM€ 35,5Total: M€ 514,8204 projects€20.000.000€0FP7-SPACE-2007-1FP7-SPACE-2009-1FP7-SPACE-2010-1FP7-SPACE-2011-1FP7-SPACE-2012-1
FP7-SPACE-2012-1evaluation results120100109evaluated proposalsEvaluated proposalssuccessful Selected proposals806049402018 2028110GMES SSF CC
2. Understanding theevaluation process•Evaluation roadmap for the callFP7-SPACE-2013-1•Evaluation process•ESA input
2. Understanding the evaluation processEvaluation roadmap forthe call FP7-SPACE-2013-1• Call publication: 10 July 2012• Call closure / submission deadline: 21 November 2012at 17:00:00• Selection of experts: June- Sept 2012• Eligibility checks andallocation of experts to proposals: December 2012• Evaluation: December 2012 –February 2013• Information to coordinators: March 2013• Start of negotiations: April 2013
2. Understanding the evaluation processSelection procedure -Main actors• Experts: evaluate proposal in an individualmanner and then agree on marks and commentsduring consensus meetings• Moderators: REA/EC staff moderating discussionsduring consensus meetings• Rapporteurs: read but do not evaluate proposals,draft consensus reports, support theharmonisation process (consistency betweencomments and marks)• Panel of experts: examine and compare consensusreports in a given area, check consistency,recommend priority order of proposals ifnecessary• Independent observers: assess the evaluation andreport on the conduct & fairness of the evaluation
2. Understanding the evaluation processOverview FP7 selectionprocedure“remote”using IT toolBrusselsBrusselsSubmissionIndividualreadingConsensusPanelFinalisationFull ProposalProposalformsEvaluatorsCriteriaEvaluatorsRapporteurModeratedby REAEvaluatorsModeratedby REACriteriaFinalranking listRejection listEligibilityCriteriaProposals insuggestedpriority orderREAExternal expertsEC
2. Understanding the evaluation processEvaluation procedurefor each proposalIndividualreadingProposal Xcopy 1Proposal Xcopy 2IERexpert 1IERexpert 2ConsensusmeetingCR(+EIR?)3 expertsProposal Xcopy 3IERexpert 3Note: There may be more than 3 expert evaluatorsIER=Individual evaluation reportCR=Consensus ReportEIR=Ethics Issues Report
2. Understanding the evaluation processGuiding principlesObjectivityo Each proposal is evaluated as it is writtenTransparencyo Evaluation experts make their judgment against theofficial evaluation criteria, and nothing elseConsistencyo The same standard of judgment is applied to eachproposal (including resubmitted proposals)
2. Understanding the evaluation processSelection of experts• Minimum 3 experts per proposalInitial selection of experts based on topics of the work programme(June to September 2012) - taking into account geographical andgender balance, types of organisation, broad coverage of areas ofexpertiseFinal selection of experts based on proposal received (end November)- taking into account expertise and avoiding conflict of interest (e.g.remove experts if their organisation participate to a proposal)Experts mainly from EU MS and associated countries, but can be anyother country (eg USA). Each evaluation 40 to 50% of new experts• If you want to become an evaluation expert yourselfregister through the participant portalhttps://ec.europa.eu/research/participants/portal/page/expertsWhen describing your areas of expertise, use the keywords of the workprogramme and keep your profile up-to-date!
ESA input• DG ENTR works closely with ESA on thedevelopment of the Space policy and itsimplementation through FP7• ESA does not participate in FP7 Space projects• ESA is involved in the evaluation process byproviding comments on potential duplication withtheir programmes.• Finally, ESA may provide experts with the role ofreviewers for running projects
3. Key highlights on proposapreparation•Eligibility•Evaluation criteria•Additional explanations andhints
3. Key highlights on proposal preparationEligibilityA proposal is eligible if:1. Received before the deadline (seconds count!)2. Minimum conditions for participation metooAt least 3 independent legal entities from 3 Member states orAssociated countriesOther conditions in the work programme met(e.g. maximum requested EU contribution respected, presenceof SMEs)3. CompleteoRequested administrative forms+ proposal description (readable, accessible and printable)4. (At least partially) in scope of topic(s) and funding schemeIneligible proposals are not evaluatedIneligibility can also be discovered later during evaluation
3. Key highlights on proposal preparationEU Member States andFP7 Associated CountriesEU Member States:Austria, Belgium, Bulgaria, Cyprus, Czech Republic,Denmark, Estonia, Finland, France, Germany, Greece,Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg,Malta, Netherlands, Poland, Portugal, Romania,Slovakia, Slovenia, Spain, Sweden, United Kingdom.FP7 Associated Countries:Albania, Bosnia and Herzegovina, Croatia, FaroeIslands, FYR Macedonia, Iceland, Israel, Liechtenstein,Montenegro, Norway, Serbia, Switzerland and Turkey.
3. Key highlights on proposal preparationInternational cooperationThird country participants• Third countries may participate in FP7 activities• If the minimum conditions in FP7 ‘Rules for Participation’ & the WorkProgramme are met.1) International Cooperation Partner Countries (ICPCs) (complete list:ftp://ftp.cordis.europa.eu/pub/fp7/docs/icpc-list.pdf).- eligible to receive EU funding.- they can be the coordinator2) High-income countries- all the countries not included in the ICPC list and not associated to FP7.- e.g. USA, Canada, Japan, Korea, Singapore, Australia and New Zealand.- eligible to receive EU financial contribution if certain conditions are met:1. if funding is provided for in a bilateral scientific agreement or2. if clear provisions are included in the work programme/call for proposalsNB: NASA has signed a specific agreement with EC for FP7 Space Programme
3. Key highlights on proposal preparationEvaluation criteria1. Scientific and/ortechnologicalExcellence(relevant to thetopics addressedby the call)2. Quality and efficiency of theimplementation and themanagement3. The potential impactthrough the development,dissemination and use ofproject resultsCollaborativeprojects(NO CSA in2013 Call)•Soundness ofconcept, and quality ofobjectives•Progress beyond thestate-of-the-art•Quality andeffectiveness of theS/T methodology andassociated work plan•Appropriateness of themanagement structure andprocedures•Quality and relevantexperience of the individualparticipants•Quality of the consortium as awhole (includingcomplementarity, balance)•Appropriateness of theallocation and justification ofthe resources to be committed(staff, equipment,…)The evaluation expertsuse assessment formswith these questions•Contribution, at theEuropean and/orinternational level, to theexpected impacts listed inthe work programme underrelevant topic/activity•Appropriateness ofmeasures for thedissemination and/orexploitation of projectresults, and management ofintellectual property.
3. Key highlights on proposal preparationEvaluation criteriaOr in other words ….
3. Key highlights on proposal preparationWhat are the experts looking forScientific and/or technological Excellence• Objectives should be focused (what do you wantto achieve in this project?) and clearly linked tothe call topic.• Progress beyond the State of the Art should beup-to-date!(be careful if resubmission...)• Methodology and work plan: include risk analysis& contingency plans
3. Key highlights on proposal preparationWhat are the experts looking forImplementation & Management• Management structure & procedures:Include decision making & conflict resolution• Quality & relevant experience of individual participants• Quality of the consortium: complementarities & balance• Allocation of resources: should be explained and justifiedo How is the totality of the necessary resources mobilisedo Will the resources be integrated in a coherent wayo Is the overall financial plan adequate for the project• Subcontracting: no core tasks - no predefined subcontractorsMust be described and an estimated budget providedIf third parties involved described their role !
3. Key highlights on proposal preparationWhat are the experts looking forImpact, Dissemination and IPR• Contribution to the expected impact listed in thework programme under the relevant topic/activity• Dissemination and exploitation of project resultsHow this will increase the impact?What are the targeted audiences of the activities?• Management of intellectual property:Which existing knowledge is needed for the project?How this existing knowledge will be available to thepartners?
3. Key highlights on proposal preparationScores and meaning• 0 - The proposal fails to address the criterion under examinationor cannot be judged due to missing or incomplete information• 1 - Poor. The criterion is addressed in an inadequate manner, orthere are serious inherent weaknesses.• 2 - Fair. While the proposal broadly addresses the criterion, thereare significant weaknesses.• 3 - Good. The proposal addresses the criterion well, althoughimprovements would be necessary.• 4 - Very good. The proposal addresses the criterion very well,although certain improvements are still possible.• 5 - Excellent. The proposal successfully addresses all relevantaspects of the criterion in question. Any shortcomings are minor.Scores with half pointsare possible Individual criteria threshold: 3Total score threshold: 10
3. Key highlights on proposal preparationPresence of SMEs• The expertise of the SMEs should be clearly highlightedand their work well linked to the activities: part of thequality of the consortium• Can be an added value for the exploitation of resultsand thus the impact• Potential criterion taken into account if prioritisation isneeded between proposalsRule: no special evaluation criterionException: Topic 3.1-1: mandatory SME participation!
3. Key highlights on proposal preparationGender aspects• In the consortium, promote mixed teams at all level(Work Packages, Work Package leaders, steeringcommittees, advisory boards...)• Check if gender may be a factor to be taken intoaccount in your research (e.g. users)• Proactive activities to promote gender equality in yourproject or field of research (e.g. specific events)• Gender “toolkit” available athttp://www.yellowwindow.be/genderinresearch/
3. Key highlights on proposal preparationEthics issues• Highlight in the proposal any ethics issues, e.g.- data protection issues (privacy)- Dual use: research having direct military use or the potential forterrorist abuse• New enhanced ethics issues procedure Rules for submission, evaluation, selection, awardPre-screening by evaluators“Flagged” proposals go through Ethical Review(body of independent ethics experts)= > implementation of requirements at negotiation stage
3. Key highlights on proposal preparationPut yourself in the shoesof the experts• Understand the work programme topic and address its essential spirito A proposal fully out of scope is ineligibleo A proposal partially out of scope will score low• Respect requirements and instructions in the work programme (e.g.maximum amount of requested EU funding, SME participation)and check the Guide for Applicants (e.g. page limits).• Proposers need to convince the evaluation expertso They are under time pressure during both the individual evaluationand the consensus meetingo Make it easy for them to find the answers in your proposals to thequestions to assess the proposal against the evaluation criteriao They must justify their marks, give them evidence.o Avoid inconsistencies in the proposal to make a good impression.o Evaluators are instructed to look at the substance, not thepresentation, but a careful presentation helps.
Where to find relevant informationRelevant reference documentsFP7 Legal basis documents generally applicableDecision on the Framework ProgrammeRules for Participation http://cordis.europa.eu/fp7/find-doc_en.htmlLegal documents for implementationRules for submission, evaluation, selection, awardREA grant agreementRules on verification of existence, legal status,operational and financial capacityGuidance documentsEthics issues:Negotiation Guidance NotesGuide to Financial IssuesEthics check listSupporting documentsGuide to IPRChecklist for the Consortium AgreementTemplates for Description of WorkCall documents: Participant portal
Where to find relevant informationRelevant information sourcesParticipant Portal: http://ec.europa.eu/research/participants/portal/CORDIS: http://cordis.europa.eu/fp7/participate_en.htmlMore background documents onhttp://ec.europa.eu/enterprise/policies/space/researchCall-specific questions (up to 2 weeks before deadline):email@example.comGeneral FP7 related questions:http://ec.europa.eu/research/enquiriesSpecialised and technical assistanceeFP7 Service Desk: http://ec.europa.eu/research/participants/portal/page/contactusEPSS Help desk: firstname.lastname@example.orgIPR help desk: http://www.ipr-helpdesk.orgEthics help desk: http://cordis.europa.eu/fp7/get-support_en.htmlNational Contact Points: http://cordis.europa.eu/fp7/ncp.htm
Thank you for yourattention!REA – Research Executive Agencyhttp://ec.europa.eu/research/rea COV2 ▪ 1049 Brussels ▪ Belgiumsite 16 Place Rogier ▪ 1210 Brussels +32-2-299 11 11 +32-2-297 96 09Copyright © 2007-2009 Close Comfort