Evaluating Country Programmes - OECD Online Bookshop

OECD 1999
Country Assistance Evaluation: Case Study of the OED Philippines
met with government officials on occasion of their visits to Washington for the spring
meetings. The field visit in June 1997 canvassed views through individual interviews
from former and current government officials, members of Parliament, academics,
business people, foreign donor representatives, and civil society. Two roundtables
organised in Manila allowed an opportunity for one-stop discussions with Filipino
officials involved in decentralisation issues and with NGO representatives.
Methodological challenges
We faced three main methodological challenges: counterfactuals, attribution
and metrics. There were no clear performance benchmarks set in the Philippines
CASs under review (unlike in the most recent CASs), nor stark alternatives to the
adopted assistance strategy. The CAR, however, noted that the net resource transfer
had been largely negative every year since 1986, and that the pace of reforms might
have been faster with less stringency by the Bank. We also tried to inject frequent
comparisons of the country achievements with its neighbours, mainly on the direction
and degrees of improvements rather than on absolute values. OED’s ambitions
and methodological approaches with respect to the counterfactual issue are still
under debate. Work is also going on at the Bank to construct a comprehensive
scorecard, with explicit goals for a number of key economic and social indicators, in
the context of the new development framework, that may provide an automatic
counterfactual against which to compare performance.
Relating country performance to Bank assistance is another difficult topic, especially
when the Bank’s influence on policy reform is shared with the Fund and Government,
when its lending role is overtaken by international capital markets and other
donors [Japan and the AsDB (Asian Development Bank) in the case of the
Philippines], and when there are numerous external shocks. In the case of the
Philippines CAR, the simplistic approach taken was to link observed outcomes to the
Bank’s stated ambitions. So, as in the case of the positive outcome of the structural
adjustment reform programme, as the Bank pushed in the right direction and offered
adequate support, the CAR rated the outcome of its assistance as satisfactory. The
CAR, however, did not make an effort to determine the relative credit among the
Fund, the Bank and the Government. And similarly for the indispensability of the
Bank’s role: it could have very well been the case that, even without the Bank, the
country could have followed the same reform path, relying on other advisors (e.g., IMF,
International Monetary Fund; and HIID, Harvard Institute for International Development).
It would clearly be easier to deal with this issue if the major donors were to
move towards preparing joint CASs and country evaluation reviews.
The old generation Philippines CAR, unlike our latest country evaluations, was
allowed to be a bit loose in its ratings (“Bank assistance… was satisfactory but
uneven and below potential”). It skirted the explicit rating of its impact on
211