Evaluating Country Programmes - OECD Online Bookshop

Evaluating Country Programmes
28
In most cases it will be very difficult to put real figures to aid impact at the
country level. The World Bank, whose assistance perhaps lends itself to quantification
more than that of most donors, has stated that it is only possible to estimate
an economic rate of return for one-third of all Bank projects. However, if it is impossible
to measure or even trace in detail all the complex effects of aid (intended and
unintended, positive and negative, short- and long-term), it is still possible to
obtain some reasonably good impressions of impact: a qualitative, indicative, reasoned
statement about the links that are seen to exist between aid and observed
changes. Impact here is seen as lying only a little beyond effects (the immediate
and quantifiable outcome of intervention), although obviously this varies depending
upon the type of aid activity.
Two different broad approaches to Country Programme Evaluation started to
emerge from the conference. One looks at individual projects and aggregates the
findings, and the other at the country programme as a whole. The former is essentially
additive, using more conventional evaluation techniques to look at a representative
selection of projects; the latter is basically subtractive, looking at the
overall achievements of the country and attempting to explain what part of national
success and failure can be attributed to the influence of the donor’s aid.
Which of these two approaches is adopted depends largely on context. The
former works best when the country programme is small in relation to national GDP,
to total capital flows and/or to other donors. When the donor in question clearly
dominates aid to the partner in question and aid is a major factor in the partner
economy, then the second, broad-brush approach may be more rewarding. This was
the case in, for example, the study of USAID’s development assistance to Costa
Rica. The overall argument that USAID had made a difference was based upon
broad comparisons (with the situation in the 1950s, with other countries in the
region, and with forecasts made in the early 1980s before a new phase of American
aid began). Change (the before and after) and comparator-based and ex-ante counterfactuals
(models of with-and-without) were used to support a general claim that
assistance had made a difference. 16 Examination of particular components of the
aid relationship was then used primarily to find explanations for the general impact
– on growth, integration to global information flows, living standards – that emerged
at the national level and in the sociological studies of change in two study villages.
The USAID/Costa Rica case study is somewhat unusual with regard to impact
evaluation. Few bilateral donors will dominate assistance to a given country to the
extent that USAID did in Costa Rica. Most will have to approach Country Programme
Evaluation by looking at aid impact in a number of discrete sectors or geographical
regions in which they work. In this more common situation, it is also necessary to
look at more intangible, non-project impacts (influence upon policy exercised in
informal, unwritten ways through relationships built up over a long period), but the
impact of aid cannot simply be read off from change in national performance
OECD 1999