The Matrix System at Work - Independent Evaluation Group - World ...

Appendix C
Responsiveness and Effectiveness of the
Bank’s Country Programs
1. Are Bank country programs responsive and effective in meeting country needs?
For the last eight years, the Bank has been issuing Country Assistance Strategy
Completion Reports (CASCRs)—self-assessments of country program achievements
at the completion of the Country Assistance Strategy (CAS) period—and the
Independent Evaluation Group (IEG) has conducted CASCR reviews to validate
their accuracy and draw additional lessons from the experience of CAS
implementation. These reviews show that outcomes, measured as achievement of
objectives by CAS pillars, were satisfactory less than two-thirds of the time. As the
number of completed CASCR reviews now approaches 100, it is possible to take a
closer look at specific factors that underlie the aggregate results.
2. IEG reviewed 96 CASCR reviews—all those available as of December 2010 and
which reviewed CASs prepared after the introduction of the matrix system—
covering country programs in 80 countries. IEG developed an aggregate measure of
the two goals of the 1997 reform decomposing the quality of country programs into
eight component variables—four that reflect responsiveness and four representing
effectiveness. Responsiveness encompasses the most basic measure—alignment with
a country’s national priorities—and includes also a measure of ownership of the
program by the government and other stakeholders including civil society, realism of
the objectives and planned instruments, and flexibility to adapt to changing country
circumstances. Effectiveness is measured by the quality of the results framework of the
CAS, the quality of portfolio implementation of the program, outcomes of projects and
operations, and outcomes of analytical and advisory activities (AAA).
3. Each component was rated separately on a four-point scale —Unsatisfactory,
Moderately Unsatisfactory, Moderately Satisfactory, or Satisfactory—using only
evidence presented in IEG’s CASCR reviews. An aggregate rating was arrived at for
each country program, applying equal weight to each of the eight components.
Ratings for each dimension and in aggregate were compared across Regions, income
categories, and two time periods (CAS periods beginning between FY98 and FY03
and those beginning between FY04 and FY08).
4. How does responsiveness and effectiveness vary by Region? What drives Bank
performance in each of the eight components? What are the primary deficiencies in
areas of poor performance? This appendix explores the underlying factors driving
the ratings for alignment with national priorities, ownership, realism, and flexibility
on the responsiveness side, and results frameworks, quality of portfolio
124