Evaluating Country Programmes - OECD Online Bookshop

Country Assistance Evaluation in the Multilateral Development Banks
There are a number of other issues that need to be considered in country assistance
evaluation. Foremost, it must be decided what is to be evaluated. Using the Logical
Framework Analysis approach, MDB’s activities in a country typically entail
defining a country strategy or programme, usually consisting of a strategic goal or
perhaps several goals and a series of clearly defined objectives that contribute to
the goal (s). These goals and objectives can then be translated into a list of operations
and other activities. In addition to customary investment loans and technical
co-operation (TC), other activities include balance of payments support, support
for economic stability, structural adjustment and sector adjustment loans, provision
of advice and policy dialogue either directly through its operations or through economic
sector work (ESW) – debt management support and services, loans and
equity financing to the private sector, aid co-ordination and mobilisation of support
through cofinancing with other donors, export credit agencies and the private sector.
Such a lengthy list makes it difficult to evaluate an institution’s entire activities.
There is a need for selective focus on activities. Traditionally, investment projects
have been the easiest to evaluate, even when focusing on impact. An institution’s
sectoral impact, although more difficult to evaluate, can still utilise benchmarks and
indicators to assess overall results (i.e. share of households with electricity). At the
macro-economic level, results are much more difficult to measure and attribute.
In addition, it often happens that activities are not recounted in country programme
results and other official documents for which an institution should receive
credit. These activities include policy advice sought and received by the country,
which it eventually pursues or adopts, often leading to reform and growth, yet not
attributed to the MDB. Other examples include projects or programme proposals
from the MDB that the country decides to fund on its own initiative or resources
obtained through aid co-ordination or mobilisation that help the country to
develop. Very few, if any, of these forms of assistance have been evaluated so far.
On the other hand, an institution may have spent considerable time, effort and
resources in project planning and preparation, only to find that the country cancels
the project before implementation or midway through the implementation process.
In this case, the efforts made by the MDB are listed as inputs, but bear no results
or impact on the country.
From the outset of the country assistance evaluation, the subject and the
scope of the exercise should be made clear. A country strategy evaluation (CSE) –
the evaluator’s unit of account – can be limited to evaluating simply the relevance
of the institution’s strategy to a country. A CPE can focus on the aggregate outcome
and effectiveness of the loans and TC activities only, and choose to ignore all the
other activities. A country assistance evaluation (CAE) can attempt to evaluate all
aspects of an institution’s assistance to a country, but it may miss important elements
that fail to be reported.
OECD 1999
147