Chapter 2. Evaluation of research projects, « ex-ante ».

Guidelines.Chapter 2.Sub-Chap. 2.1 Eval.proj.ex-ante. Draft 12.8.10
Chapter 2. Evaluation of research projects, « ex-ante ».
(In this document, italics between parentheses concern sentences or section aimed at the Subgroup
and Task Force members, and will not be included in the final guidelines.
The present chapter’s structure is adapted from the corresponding chapter on “ex-post”
evaluation of a research project. As it now stands, it will need much revision after the
drafting of the others).
A. General
This chapter’s content, needless to remember, is aimed at the author(s) of a specific
evaluation guide : in the present case a guide for evaluating research proposal. The specific
guide authors will pick from this material what is relevant for their needs and situation.
B. Steps
1.Check whether preliminary stage fulfilled (see Chapter 1)
2.Formulate the questions of evaluation
3.Decide which dimensions will be considered
4.Select the criteria to be used
5.Select and define procedures
6.Check whether the conditions for evaluation are satisfied, including ethical
considerations
7.Anticipate difficulties
8.Distribute responsibilities
9.Check for consistency
Step 1. Check whether the preliminary stages have been fulfilled
For a list of the preliminary stages, see Chapter 1. More specifically, prior to start
constructing the local evaluation guide, four main aspects need to be examined and decided
upon :
- Definition of the evaluation objectives
- Identification of the discipline(s) concerned
- Listing of the major stakeholders
- Type of evaluation
Reprendre ici
1.1 Definition of objectives
Possible objectives are listed in Annex 1. In the present case of “ex-ante” evaluation of
projects, the most common objectives are :
- Accepting a manuscript submitted for publication

- Continuous funding (of same project; of new proposal)
- Granting a degree
- Recognition : award, prize)
- Valorisation
1.2 Identification of disciplines concerned
See Annex 2
Specify how to operate in the case of a multidisciplinary research
1.3 Listing of the major stakeholders
See Annex .3
1.4 Type of evaluation
Enounce clearly whether
- Evaluation was planned from the stage of project design on
- Participatory, or external evaluation, or both.
Step 2. Formulate the questions of evaluation
Express concisely and with simple words
- What does the evaluator want to know
- What for
(This still needs to be worked out by the Sub-group)
Step 3. Decide which dimensions will be considered
Evaluation of a research project is necessarily multidimensional. Hence the need to carefully
choose the dimensions to be taken into consideration in each specific evaluation guide.
The basic dimensions of evaluation of a development research project, “ex-post” are :
- Relevance for development
- Scientific quality
- Valorisation
- Performance
- Impact on development
Other dimensions may also be considered, depending upon the context and upon the
objectives of the evaluation. Examples :
- Innovation; originality
- Comprehensiveness
- Cost effectiveness
- Appropriation (of the results )
- Sustainability
The last two dimensions may often be given a higher weight, particularly when an applied
research project with a clear development goal will be evaluated.

See Annex 4 for details.
Step 4. Select the criteria to be used in the specific evaluation guide
(This section should explain two processes :
- the choice of criteria on the basis of the selected dimensions
- the principles and rules for giving relative weights to the criteria, when necessary.
Details to be covered here and in step 5 Procedures)
Step 5. Select and define evaluation procedures
(This section will depend on decisions made during steps 1 (sub-step 1.4) and step 4. To be
written when the guidelines and the annexes are more advanced).
See Annex 5
Step 6. Check whether the conditions for a meaningful evaluation are satisfied,
including ethical considerations
This step must be included in each specific evaluation guide. Yet the conditions of evaluation
are quite general and are often common to many situations.
The guidelines’ users will find the necessary guidance in Annex 6.
Step 7. Anticipate potential difficulties
This step will often prove helpful to the specific guide’s user, that is the evaluator himself.
Mistakes, pitfalls or drawbacks he may face will be identified, and manners of overcoming
them will be proposed. Examples are :
- Criteria in the guide are ambiguous, or difficult to interpret (because for instance the
instructions for use are not clear or not complete)
- Inadequate information on the data or on the way they were calculated
- Biases : the conditions for a good evaluation are not fully satisfied
- Lack of transparency
- Conflicts of interest. Appeal to the evaluator’s ethics is sometimes needed in order to
prevent or correct such biases
- Results of evaluation are used for a purpose which does not correspond to the stated
evaluation objectives
- Etc.
Step 8. Distribute responsibilities
The specific evaluation guide will define who, among the actors which were identified at
start, will do what.

This applies to the specific guide authors as well as to the actual evaluators.
Step 9. Check for consistency
Return to Chapter 1 (Preliminary stages) and make sure all the indications contained in
Chapter 1. have been given due consideration. That does not mean, obviously, that all have to
be adopted. Proceed iteratively.
________________________________________