Evaluating Country Programmes - OECD Online Bookshop

Evaluating Country Programmes
54
It should be emphasised that the study can only provide a preliminary overview.
First, the field of country programme evaluations is still quite new: lessons
and pointers towards best practice are only starting to emerge (IDB, 1998: ii). Secondly,
the information base for our findings is relatively small. This is partly
because there is only a small population of CPEs (compared to other forms of evaluation);
partly because we have only been able to obtain about two-thirds of those
CPEs we know to exist; and partly because there has not been enough time to
review thoroughly all of those we have received.
On the basis of a broad definition of what constitutes a CPE, we have
identified 110 in existence (14 dating from before 1994 and 96 produced between
1994 and the present). These 110 are summarised as a matrix of donor and partner
country in Table 2.1 and listed in full (by title, donor, partner country and year) in
Appendix 2.1. We have been able to obtain 67 of these (of which 58 date from 1994
or later). Of these, we were able to review 25 in the time available to us. A column
in Appendix 2.3 notes which reports were included in the sample.
Even allowing for the fact that recent reports are easier to obtain than old
reports, it is clear that there is a trend among donors towards more country programme
evaluations. This is illustrated in Figure 2.1.
Given i) the time available to us and ii) the nature of the CPEs as institutional
products which are somewhat standardised within any given donor, we selected our
sample purposively rather than randomly, prioritising the lengthy and in-depth
CPEs of the major actors, but including also a selection of shorter studies. Each of
the 25 CPEs we reviewed was summarised by means of a short coding sheet. This is
reproduced as Appendix 2.4.
The presentation of our results needs some discussion. We do not claim that
our findings are statistically representative. Instead, having covered a representative
sample of CPEs, we are confident that we can present a reliable picture of the
range of approaches and experiences in contemporary CPEs. While we have
included analysis of what proportion of CPEs reviewed can be said to fall into one
category or another for any given criterion (presented as tables), we have also
sought to provide examples of what these categories mean in practice (summarised
in boxes).
Apart from the country programme evaluation reports themselves we drew on
three different kinds of material in identifying themes and issues in contemporary
CPE practice:
– Papers prepared about the art-form of CPEs. This included the conclusions
of the first Vienna meeting, manuals or guidelines by donors (Japan, UNDP)
review papers on the subject (IDB, WB), and background papers produced
for specific donor exercises.
OECD 1999