Evaluating Country Programmes - OECD Online Bookshop

Evaluating Country Programmes
158
Measuring efficacy and effectiveness
Efficacy is typically measured by determining whether the assistance
instruments achieved the objectives of the strategy. The Asian Development
Bank (AsDB) has taken this one step further by distinguishing between efficacy
– which it defines as progress in achieving the Bank’s macro-economic and sectoral
objectives for the country through its operations – and effectiveness which
it defines as achievement of objectives of individual Bank lending and nonlending
operations. The World Bank CARs and CANs mainly review adjustment
and macro-economic-based operations, sector support activities, ESW and the
Bank’s role in aid co-ordination of the assistance activities. These instruments
account for the bulk of assistance the World Bank has provided in its assistance
programme to the selected countries.
Efficacy is also measured on a sector basis. Conclusions occasionally are
reached for all assistance activities and instruments within a sector, more often they
are measured over a period of time (Zambia, Argentina and Côte d’Ivoire). In other
cases, efficacy is measured and reported on a sector-by-sector basis and for the different
types of instruments (Morocco, Poland and the Philippines). In some cases
the evaluation did not provide a rating of efficacy, given that the programme was
ongoing (Mozambique and Albania). In these cases, implementation experience
was reported.
Efficacy refers to the extent to which the assistance instruments have achieved the intentions
and objectives set.
Measuring efficiency
The World Bank CARs and CANs measure efficiency by comparing the costs of
the preparation and implementation of the country operations and programme to
that of other country’s programmes – either in staff years or in direct dollars. This
comparison can be made with the Bank’s or the region’s average, or with similar
countries in the region. Sometimes efficiency is measured in terms of the length of
time between approval of the project and its implementation. The World Bank has
begun to explore how to compare the costs per unit of benefit, which would be a
better way of measuring efficiency although the difficulty here is in defining benefit.
While it is relatively easy to assess the costs of a strategy and related instruments
(staff weeks or dollars spent on preparation, supervision and implementation of a
programme), the unit of benefit cannot be defined so easily at country level. Further
effort needs to be made in this area.
Efficiency refers to whether the method and procedures employed to arrive at a country programme
and strategies and the instruments used to implement it have been the most cost-efficient.
OECD 1999