Evaluating Country Programmes - OECD Online Bookshop

Traditional tools: relevance, efficacy and efficiency
OECD 1999
Country Assistance Evaluation in the Multilateral Development Banks
The MDBs have traditionally attempted to measure relevance, efficacy and
efficiency of projects and programmes. The same criteria have been used to evaluate
country programmes and strategies. They may be broadly defined as follows:
– Relevance – Whether the strategy, goal (s), objectives, activities and operations,
policy formulation, economic sector work and aid mobilisation and coordination
have addressed, and been consistent with, the needs of the country.
How these relate to the institution’s mandate, the country’s internal strategies,
priorities and policies as well as those of other donors and investors may
also influence the relevance.
– Efficacy – Whether a country programme has been effective in delivering
results through its loan, technical co-operation, policy dialogue and other
activities compared to stated intentions and objectives.
– Efficiency – Whether the method and procedures employed to arrive at a
country programme and strategy and the instruments used to implement it
(loans, TC, policy dialogue, ESW, aid co-ordination, etc.) have been the most
cost-efficient.
Measuring relevance
In the studies completed by the World Bank, relevance is measured by first
examining the needs of the country over a period, normally at a macro-economic
level and sometimes at the sector level. Successively, it is assessed by reviewing the
Bank’s assistance to determine whether a correct diagnosis was made; whether the
objectives of the strategy were consistent with the diagnosis; and if the assistance
and instruments were adequate for achieving those objectives. On the whole, the
Bank chose countries for their country assistance reviews (CARs) that had experienced
substantial amounts of structural adjustment, sector adjustment, stabilisation
or debt reduction funding. This facilitated a macro-economic assessment of the
needs of the country, rather than a more detailed sector assessment of its needs. It
also facilitated the assessment of the policy dialogue and ESW that the Bank undertook
during that period. In almost all of the CARs and shorter country assistance notes
(CANs) the strategy was judged to be relevant. The Zambia CAR concluded that the
Bank’s strategy was less relevant to the country’s long-term development than it
could have been. The Argentina CAR found the strategy to be relevant, but not the
instruments. Some CARs (Argentina, Côte d’Ivoire and Bangladesh) found the country
strategy to be more relevant in one period than another. In a number of cases (Ghana
and Mozambique) the CAR chose not to give relevance rating to the strategy.
Relevance examines whether the institutions’ strategy, goal (s), objectives and activities have
addressed and been consistent with the needs of the country and how they relate to other priorities.
157