The Matrix System at Work - Independent Evaluation Group - World ...
The Matrix System at Work - Independent Evaluation Group - World ...
The Matrix System at Work - Independent Evaluation Group - World ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
CHAPTER 4<br />
DELIVERING QUALITY SERVICES<br />
4.7 In addition to assessing the outcomes of oper<strong>at</strong>ions after they have closed,<br />
IEG assesses Bank performance in ensuring quality <strong>at</strong> entry and the quality of<br />
supervision. Supervision is more crucial for investment lending projects, which are<br />
typically active for six to seven years, than for development policy oper<strong>at</strong>ions which<br />
are typically closed within a few months of approval. For this reason, the analysis is<br />
restricted to investment lending projects.<br />
4.8 <strong>The</strong> quality of supervision and quality <strong>at</strong> entry have shown a declining<br />
trend since the introduction of the m<strong>at</strong>rix. <strong>The</strong> quality of supervision peaked <strong>at</strong> 86<br />
percent moder<strong>at</strong>ely s<strong>at</strong>isfactory or better during FY97-99, falling to 78 percent<br />
during FY04-06, as shown in Figure 4.2, a difference th<strong>at</strong> is st<strong>at</strong>istically significant <strong>at</strong><br />
95 percent confidence. Bank performance in ensuring quality <strong>at</strong> entry peaked <strong>at</strong> 74<br />
percent during FY96-98, falling to 67 percent during FY04-06, but this decline is not<br />
st<strong>at</strong>istically significant.<br />
Figure 4.2. Investment Lending Quality and Outcomes by Fiscal Year of Approval<br />
Share of moder<strong>at</strong>ely s<strong>at</strong>isfactory or<br />
better (3‐yr centered avg)<br />
90%<br />
85%<br />
80%<br />
75%<br />
70%<br />
65%<br />
60%<br />
55%<br />
50%<br />
Quality of supervision<br />
Quality <strong>at</strong> entry<br />
Introduction of<br />
m<strong>at</strong>rix system<br />
Outcomes<br />
Source: IEG evalu<strong>at</strong>ions (available in Business Warehouse)<br />
4.9 Three indic<strong>at</strong>ors offer additional clues to the quality of the portfolio, based on<br />
the inform<strong>at</strong>ion made available in real time to task teams and managers. Without<br />
candid inform<strong>at</strong>ion on the likelihood an oper<strong>at</strong>ion will meet its objectives, Bank<br />
managers and government implementing agencies cannot alloc<strong>at</strong>e <strong>at</strong>tention and<br />
resources to where they are needed. <strong>The</strong> net disconnect compares self-r<strong>at</strong>ings to<br />
ultim<strong>at</strong>e r<strong>at</strong>ings in independent evalu<strong>at</strong>ions: it is the difference between the<br />
percentage of projects r<strong>at</strong>ed as moder<strong>at</strong>ely uns<strong>at</strong>isfactory or worse by IEG and the<br />
percentage r<strong>at</strong>ed as such by the Regions in the final Implement<strong>at</strong>ion St<strong>at</strong>us and<br />
64