Evaluating Country Programmes - OECD Online Bookshop

Evaluating Country Programmes
36
Box 1.7. Recommendations for good practice
in Country Programme Evaluation
– The terms of reference (ToR) for a CPE should clearly state the purposes (input to
country planning) or provide a more general agency review of “lessons learned”);
the timing; the criteria to be used; and the ways in which the findings are to feed
into policy or practice (approval, dissemination and follow-up procedures).
– The ToR should clearly specify the goals against which country programme success
or failure must be judged (e.g. poverty reduction, growth in certain sectors).
These goals may be those of the individual projects; of the country programme (if
a country strategy is in existence); or of the agency as a whole. If the CPE is to
examine more than one level of goals, or if goals have changed over time, it
should address specifically the complementary or contradictory relationships
between goals at different levels.
– The terms of reference for CPEs should generally require the evaluators to
consider the influence of systemic issues i.e. policy and administrative
constraints affecting the programme, on both the donor and recipient sides.
Appropriate expertise, time and access should be allotted to these tasks.
– The identification of valid comparators, essential to all evaluation, is harder
at the country programme level. If a number of donors were to carry out CPEs
jointly, then this would provide each with benchmarks against which to judge
their own performance. Even if it is hard to obtain a valid comparator for the
full country programme, evaluators should seek to obtain programme, sector
or project evaluation reports from other donors. This will allow at least some
elements of country programme performance to be benchmarked.
– Detailed analysis of impact is always desirable but may be beyond the scope of
some CPEs. Donors should design CPEs on the basis of what is feasible and
justifiable in a given situation: when impact evaluation is impossible or can only
be obtained at a prohibitive cost, this should be acknowledged and the goals of
the CPE limited, realistically, to a rigorous examination of the relevance,
efficiency and sustainability of the country programme.
– Mechanisms to strengthen donor-recipient partnership include: consultation
prior to design of ToR; partner-country or joint partner-donor advisory groups;
presentation of draft findings to stakeholder groups for review; and invitations for
a formal partner reaction to draft and final documents. Whoever manages the
CPE, the team carrying out the study should consist of nationals of both the donor
and recipient countries.
– The widest possible stakeholder consultation and involvement is required in CPEs.
In addition to informal interviews, consideration should be given to using customer
surveys, focus group discussions and/or field-based participatory methods.
– Reporting and follow-up procedures should be specified in advance. Normally, CPEs
will be discussed either by senior management, by the governing body of an institution,
or by the donor Parliament. There is a strong case for circulating them to other
country programmes run by the donor. Unless there are specific and compelling reasons
to do otherwise, they should also be made available to the taxpaying public,
other donors working in the same country and implementing partners such as NGOs.
OECD 1999