The Matrix System at Work - Independent Evaluation Group - World ...

CHAPTER 4
DELIVERING QUALITY SERVICES
Results report. The Bank has never met its target of 5 percent or lower. The
disconnect has been at least 14 percent in every year since FY07.
4.10 The realism index compares self-ratings to recent history: it is the ratio of the
percentage of actual problem projects in the active portfolio and the percentage of
operations with moderately unsatisfactory or worse outcomes as rated by IEG for
the most recent completions (1,000 evaluations), on a rolling basis. The Bank has set
a target of 70 percent for its realism, and despite improvement over the past decade,
the target has only been met once (in FY09) in the past 10 years
4.11 IEG ratings of the quality of monitoring and evaluation (M&E) in operations
assess the extent to which M&E systems were appropriately designed, implemented,
and utilized and serve as a measure of the real-time feedback available for project
management. These ratings were fully introduced for operations exiting the
portfolio beginning in FY05. A high quality rating would indicate an M&E system
with a strong influence on project performance. M&E systems have been weak
during the entire period for which data are available, with just 25 percent rated of
substantial or high quality among operations closing in FY10, and never more than
36 percent reaching that threshold. The Bank has not identified a target for this
rating, but compared to the other dimensions assessed by IEG highlighted above,
these ratings are very low.
4.12 One last indicator—the proactivity index—measures the extent to which the
Bank has taken action when problems arise, and is defined as the proportion of
projects rated as actual problem projects 12 months earlier that have been upgraded,
restructured, suspended, closed, or partially (20 percent plus of commitments) or
fully canceled. This index has followed a downward path for a decade, dropping to
60 percent in FY01. It has not met its target of 80 percent since FY08.
Institutional Incentives for Quality
INSTITUTIONAL INCENTIVES FOR CLIENT ORIENTATION AND TECHNICAL EXCELLENCE AFTER 1997
4.13 The 1997 reforms differ qualitatively from the earlier ones in the budgetary
arrangements and introduction of dual reporting relationships. The success of the
matrix reform was predicated on the alignment between the new organizational
arrangements and accompanying incentives, and on the effectiveness of the
framework of formal and informal incentives. The new organizational arrangements
consisted of (a) reducing the number of countries covered by each country unit (thus
increasing client orientation), (b) creating networks (and Sector Boards) to link
technical sector units at the regional and Bank-wide level (to increase global technical
65