Evaluating Country Programmes - OECD Online Bookshop

Evaluating Country Programmes
164
Box 5.5. Development effectiveness index
The World Bank has formulated a process that would aggregate evaluation
results for a project into a single measure of overall project performance. The
Development Effectiveness Index (DEI) assigns a single cardinal measure for each
project which could then be used to aggregate different types of projects, sectors,
countries or regions to provide a more analytical tool to measure development
effectiveness. The indicator possesses properties for both analysis and policy formulation
that go beyond the normal financial rate of return and economic rate of
return calculations made to judge project outcomes. The formulation examines the
three result-oriented counts that are typically made for each project evaluation
undertaken:
Outcome Institutional Development Sustainability
Highly satisfactory Substantial Likely
Satisfactory Modest Uncertain
Marginally satisfactory Negligible Unlikely
Marginally unsatisfactory Unsatisfactory Highly unsatisfactory
The combination of these three scores yields 54 different possibilities for
project results, many of which would be very difficult to compare. Although
some results clearly rank a project higher in terms of success than others do, the
large number of possible results make it difficult to quantitatively rate the
project outcomes. However, after reviewing many projects, it was found that
over 80% of projects fell under 13 combinations, most of which were clearly rated
higher than others. The analysis shows that a Development Effectiveness Index
to measure overall project development impact has a manageable ordering
structure that can be used across different aggregates.
For the index the Bank chose a range between 2 and 10 in order to be able to
compare the index or indicator to other Bank performance indicators. The results
thus provide an index that allows for a better-informed comparison between individual
projects as well as a performance measure across types of projects, sectors,
countries or regions. This process allows for the aggregation of project indexes to
reach an overall single score for a country that could be used in a country evaluation
to assess outcome and effectiveness of projects and then to compare these scores
for different countries.
Approaches to country assistance evaluation
A country study or country evaluation may be considered to be a narrow
assessment or evaluation of the political, economic and/or social situation of a
country and its impact on the relationship with the donor agency. This kind of study
OECD 1999