Evaluating Country Programmes - OECD Online Bookshop

Evaluating Country Programmes
74
performance varies considerably. Some of the CPEs which were reviewed in detail
and the majority of those which were not because short, doing little more than
describing the elements of the country programme and providing some discussion
of relevance of the programme mix and the efficiency of its delivery. Among those
donors which attempted to do more, most adopted a two-level approach, describing
all sectors, channels and modalities of aid in overview and then evaluating a
representative sample of these components in more detail.
General conclusions about country programme performance may be based
upon a component-by-component analysis of the country programme portfolio and
some judgement about what proportion (of funds and projects) performed satisfactorily
or unsatisfactorily. Some of the CPEs reviewed went further, evaluating country-level
process issues (decision-making, the relevance and coherence of the
country strategy) and seeking to identify any relevant systemic issues rooted in the
relationship between the country programme and the donor headquarters. This
approach has the most to offer when the country programme itself is more than a
collection of projects. Given that few country programmes are written with an overarching
logical framework, it is necessary to examine how programme design, management
and achievements reflect evolution over time.
Finally, given the practical limitations upon those involved in a country programme
evaluation, some donors implicitly conclude that impact evaluation is a
task in which large, long-term, cross-country research projects have more to contribute
(e.g. Riddell, 1987; Cassen, 1994; World Bank, 1998). The degree to which a donor
can and should attempt to evaluate effectiveness or impact seems logically to be a
function of its evaluation resources on the one hand, and the coherence of its country
programming on the other. If a donor is small and/or spreads its aid thinly
between many countries, then it may be best to conduct relatively short and simple
CPEs which focus upon relevance and efficiency and draw logical rather than empirical
conclusions about impact. If a donor is a major actor with the resources for an
in-depth study and has progressed from token country planning (grouping together
diverse ongoing projects) to substantive country programming (programme mix
reflects systematic consideration of partner needs and donor comparative advantage),
then there is a persuasive argument for investing in impact analysis.
Summary
It is possible to draw an alternative typology based upon the scope of the evaluation
approach. Given that “scope” may be defined in several ways, several classifications
can be provided: by the timeframe that is reviewed (Table 2.5), the
degree of attention to country (external) environment (Table 2.6), or the level in the
hierarchy of performance criteria at which the CPE pitches its evaluation efforts
(Table 2.7). Of these, the third, criteria-based system seems the most useful. The
OECD 1999