Evaluating Country Programmes - OECD Online Bookshop

OECD 1999
Country Programme Evaluation: A State of the Art Review
basis for evaluation in most cases: the myriad of non-aid influences over the period
in which aid was provided makes it extremely hard to attribute change to donor
assistance. The before-after comparison may be of some use at the micro-scale (at
project level) if there is good baseline data and a comparable non-beneficiary area
or group.
It might also be possible to make some general counter-factual assumptions at
the other end of the scale under very specific (and rare) conditions: if the donor
concerned has clearly dominated aid flows to the partner country in question, if it
is reasonable to assume that no other donor would have filled its place, and if there
is enough baseline data and a good macro-economic model which would allow
some predictions of how the economy would have been likely to develop from this
base in the absence of donor aid.
Under most conditions, however, a counterfactual may simply not be possible.
The IDB has distinguished five conceptual approaches to the construction of a
counterfactual, one of which is based upon country-level comparators (see Appendix
2.7). In practice there are problems with each of these five approaches, most of
which have never been applied in CPEs. Nonetheless, they are worth attention.
Donors attempting to undertake a CPE should consider whether a counterfactual
might, in combination with other approaches, help to guide their understanding of
aid effectiveness.
In Zambia, the World Bank concluded that “Bank assistance has not turned the
Zambian economy around, but has kept GDP from falling further” (World Bank,
1996: 50). This illustrates one of the weaknesses of the before-after calculation of
country programme impact. If the country situation declines rapidly – due to
regional or global economic crisis, commodity price collapse, bad harvests or political
instability – then a country programme could be judged to have had positive
impact in cushioning the fall. But to identify and measure this kind of impact, when
indicators show stagnation or even decline, is very hard.
The themes of counterfactual comparison, attribution and partnership are
closely connected. Issues of synergy and attribution could potentially be better
dealt with through joint evaluations (World Bank, 1999: 4).
Co-operation in the CPE process: multi-donor and joint evaluations
There is a strong theoretical case for co-operation in country programme evaluation,
on grounds of both principle and practice. In reality a number of factors limit
the possibilities for joint (donor-partner) or multi-donor country programme evaluations.
There are several reasons why partnership in CPEs is desirable. First, there
may be gains in efficiency. CPEs can be expensive and time-consuming. If two or
more donors can carry out a joint evaluation for less than it would cost them to carry
81