Evaluating Country Programmes - OECD Online Bookshop

OECD 1999
Country Programme Evaluation: Synthesis Report from the Workshop
Box 1.3. Approaches to the construction of a counterfactual
1. Long-run growth models: forecasts based upon cross-sectional data and a
limited number of assumptions with regard to demographics, resources and
technology. Does not account for factors such as external shocks which might
change the pattern of growth.
2. Large-scale econometric models. Simulations are run on multi-equation
econometric models of the national economy (where such exist) to compare
performance with and without assistance.
3. Country comparisons: comparing the performance of countries with similar
baseline characteristics, one of which received aid and one of which did not
(see below).
4. Induced implicit assumptions: the evaluator uses his or her assumptions
about what explains unsuccessful operations to approximate what the situation
would be in the absence of that operation.
5. Ex-ante counterfactual: at the initiation of a country programme donor staff
should ideally prepare predictions of future country performance likely to
occur in the absence of the donor’s operations. Country performance is subsequently
compared to this and the difference attributed to aid.
(Summarised from IDB, 1999b: 4)
harder still because i) very few countries will receive no aid or aid from only one
donor; ii) there are a great number of internal and external influences upon changes
at the country level; and iii) the interaction between various causes and observed
effects are extremely complex (e.g. synergistic and sequencing effects). 14 Some
donors have nevertheless attempted such an approach (see the evaluation of
American aid to Costa Rica, Chapter 11).
An inductive approach which sought first to identify change and then to identify
the causes (aid and non-aid) which explain that change would introduce more balance
into the aid-centric picture of development that often emerges from aid evaluation.
However, it is not practical for most donors. Instead, most CPE evaluators will
of necessity concentrate their evaluation resources upon a more narrow, deductive
study working outwards (or upwards) from their own country programme activities. 15
This is the second approach sketched in Figure 1.2, described for convenience as
the “Evaluator’s perspective”. These two ideal types, describing polar ends of the
spectrum of approaches, are however useful guides: some CPEs will have a broader
approach, falling closer to the inductive end of the spectrum than others.
27