Evaluating Country Programmes - OECD Online Bookshop

OECD 1999
Country Programme Evaluation: A State of the Art Review
Table 2.6. Contents of partner country (external environment) analysis
Content of country analysis
Number of CPEs
Number %
Economic growth, structure and performance 22 88
Poverty profile (poverty line and human development indicators) 16 64
Poverty analysis (poverty processes and characteristics of the poor) 11 44
Partner government policy and politics 25 100
Aid environment 21 84
Socio-cultural background (gender, ethnicity, class) 17 56
Project area description/local needs analysis 11 44
Source: Authors.
The historical and topical aspects of “breadth” distinguished here do not seem
to be closely correlated. Some CPEs address a decades-long donor-partner relationship
but in specific, country programme -focused terms (e.g. some of the World
Bank CARs), whereas other reports include a much broader contextual analysis as
well as a long historical overview.
Depth of analysis: criteria used in evaluating country programme performance
Most, but not all, CPEs provide a clear list of criteria by which to judge success
in a country programme. The criteria most commonly used are those of efficiency,
effectiveness (or efficacy), impact, relevance and sustainability.
The definitions of these evaluation criteria and the conceptualisation of the
relationships between them draws heavily on logical framework (or logframe) analysis,
which was originally designed as a tool for project planning and evaluation, but
which has been adapted to CPEs. (The conceptual linkages between criteria in the
logical framework approach is provided in a diagrammatic form for reference in
Appendix 2.6.) Most of the CPEs we have reviewed draw upon the logframe approach
to structure their CPE process, either explicitly (e.g. IDB, 1998: 1) or implicitly.
Logframe analysis is a useful tool to assist the planning and subsequent evaluation
of a programme or project. Its value lies in that it relates criteria in a hierarchy:
if impact analysis is not feasible, then it suggests ways in which the tests of
relevance, sustainability and efficiency can provide input to the estimation of
impact. However, it is important to note that there are also a number of limitations
to a logframe approach.
i) There is considerable conceptual slippage under each of the criteria headings:
what one CPE classifies as impact another regards as effectiveness (see
Box 2.2). Some of these problems reflect translation issues (DAC, 1986: 72-3).
69