Evaluating Country Programmes - OECD Online Bookshop

Evaluating Country Programmes
24
Participatory methods which involve national partners appear to offer considerable
benefits and are embraced for practical reasons as well as a matter of general
principle. These partnership issues are addressed in more detail below.
The topic of impact evaluation was addressed by a separate group and is
reviewed below.
Impact and attribution
For more lengthy and ambitious CPEs, the key conceptual-methodological
issues are (see Box 1.4):
– How to identify and measure the impact of a donor country programme
(when aid at this scale has impact over a long term and in complex ways,
especially if one accepts that aid is intrinsically fungible).
– What yardstick should be used to come to conclusions about whether this
constitutes poor, adequate or good aid performance. Typically, this involves
a choice between i) comparing country-level impact to that achieved by
other donors in the same country or ii) comparing it to that achieved by the
same donor in another country. Neither of these may be strictly comparable.
Basic concepts in impact assessment
The study of impact involves identifying change; establishing a causal connection
(attribution) between such change and aid; and measuring the magnitude of
this change. Impact assessment is one of the largest challenges in all evaluations,
but particularly so in Country Programme Evaluation. At a project or sectoral level,
there is generally an expectation that change can be both identified and attributed
with a reasonable degree of certainty to one aid actor. At the opposite end of the
spectrum, change at the international level (for example, in progress towards
21st century targets) can be measured13 yet most donors entertain no hope of
attributing any specific part of this change to their own particular efforts. At the
country level, however, where both the partner country and the donor organisation
are used as the unit of account, there is a possibility that the impact of a specific
donor’s aid can be distinguished from that of other aid and non-aid processes.
Attribution and measurement of impact is however substantially harder at the
country level than it is for the project or sectoral level. Country Programme Evaluation
could be greatly facilitated if country-level goals for judging success are established
at the outset of a country programme (or the start of a given cycle). Having
said this, the discussion group on impact noted that there are many worthwhile aid
activities, such as informal advice or aid co-ordination efforts, which cannot be
defined precisely: there is a danger in choosing only those activities which can be
evaluated, when others may also be valuable. Similarly, it was noted that improved
OECD 1999