Evaluating Country Programmes - OECD Online Bookshop

Evaluating Country Programmes
84
With this caveat, we would suggest that it is possible to identify examples of
good practice and a movement towards a consensus regarding what constitutes the
best approach under given circumstances. We have summarised these observations
in Box 2.5 as a ten-point set of guidelines. This defines CPEs and sets out the
general principles that donors and their partners might follow. For example, we
detect strong support for CPEs to look at systemic aid management issues, on both
the donor and recipient sides, an undercurrent of enthusiasm for more joint evaluation,
and a commitment for greater partnership between donors, recipient governments,
and other stakeholders. We also find a strong commitment to use CPEs
constructively for policy formulation, and to share the results publicly wherever
possible. Less prescriptively, Appendix 2.8 provides an open checklist of questions
for those commissioning or undertaking a country programme evaluation.
The statement of good practice leaves a number of issues to be debated. Four
stand out.
First, we find several agencies struggling with the identification and attribution
of impact, whereas others do not worry too much about impact at all. It is clearly
very hard to assess the impact of a country programme, especially when the programme
a) is small in relation to total donor flows, and/or to the scale of public sector
activity and/or b) employs a diverse range of instruments (programme aid,
technical co-operation, food etc.), and channels (partner governments, NGOs,
multi-laterals, etc.), some donors have concluded that the measurement of impact
is essentially a research task, unsuitable to the time span and other constraints of
the country programme evaluation process. Others have argued that a) assessing
impact is essential, and b) some simplification is possible – perhaps by using proximate
measures of impact, stakeholder scoring or classification, or checklists. There
is still much work to do with regard to impact analysis at the country level.
Secondly, the question of scale and cost also needs discussion. Full-blown
CPEs obviously cost more than “quick and dirty” CPEs, but we found the correlation
between scope and cost to be ambiguous. The cost of a given CPE exercise (in
money and time) was generally not stated explicitly in the documents we reviewed.
However, some relatively simple CPEs appear to have been comparatively labourintensive,
while some of the more ambitious CPEs seem to have been carried out
with relatively little time input. Some CPEs were very expensive indeed, obviously
too expensive to repeat for every country.
Thirdly, there is an argument for more attention to the influence on country
programme performance of issues external to the design and management of the
country programme itself. These influences can be categorised as systemic (based in
country programme-donor headquarter relations) and holistic (the trade, investment,
strategic and historical aspects of bilateral relations which impinge upon aid
OECD 1999