The Matrix System at Work - Independent Evaluation Group - World ...
The Matrix System at Work - Independent Evaluation Group - World ...
The Matrix System at Work - Independent Evaluation Group - World ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
CHAPTER 1<br />
INTRODUCTION AND CONTEXT<br />
Assistance Evalu<strong>at</strong>ions (CAEs), CAS Completion Report (CASCR) reviews, and<br />
sector and them<strong>at</strong>ic evalu<strong>at</strong>ions to identify m<strong>at</strong>rix issues th<strong>at</strong> had a bearing on the<br />
evalu<strong>at</strong>ion questions. <strong>The</strong> methodology is described in Appendix B.<br />
1.43 <strong>The</strong> portfolio review of project outcomes and trends in the sector and<br />
them<strong>at</strong>ic composition of the portfolio are based on d<strong>at</strong>a from all 4,812 Bank lending<br />
oper<strong>at</strong>ions approved in FY1995-2010, which had a total commitment amount of $441<br />
billion. <strong>The</strong> analysis of cross support and budget support covers about half of th<strong>at</strong><br />
period, comparable d<strong>at</strong>a before FY02 being unavailable. <strong>The</strong> commitment amount<br />
for projects during this l<strong>at</strong>er period, when the fully developed m<strong>at</strong>rix system was in<br />
place, is $213 billion.<br />
1.44 Compar<strong>at</strong>ive analysis. Given the absence of baseline d<strong>at</strong>a, the evalu<strong>at</strong>ion<br />
undertook compar<strong>at</strong>ive analyses of findings from the universe of CASCR reviews to<br />
assess the responsiveness and effectiveness of country programs; time series d<strong>at</strong>a on<br />
budgets, cross support, portfolio outcomes, and them<strong>at</strong>ic composition of the<br />
portfolio; incentives and constraints of regional m<strong>at</strong>rix systems; d<strong>at</strong>a on sector<br />
staffing; d<strong>at</strong>a from manager interviews, the staff survey, and focus groups; and<br />
perceived strengths and weaknesses of the m<strong>at</strong>rix system compared with those of<br />
the pre-1997 system.<br />
LIMITATIONS OF THE EVALUATION<br />
1.45 Attribution of impacts to the m<strong>at</strong>rix is especially problem<strong>at</strong>ic in this case<br />
because of the overlapping effects of other elements of the renewal str<strong>at</strong>egy th<strong>at</strong><br />
have continued to evolve in parallel.<br />
First, most country directors were decentralized to the field (40 of 45 were<br />
decentralized by 2010), which had a significant bearing on the effectiveness of<br />
the m<strong>at</strong>rix.<br />
Second, business processes for managing budget and HR were integr<strong>at</strong>ed<br />
through SAP software in 2001, enabling system<strong>at</strong>ic storage and retrieval of<br />
d<strong>at</strong>a from FY02 but gener<strong>at</strong>ing unanticip<strong>at</strong>ed constraints on oper<strong>at</strong>ional units.<br />
Third, in 2006 two new networks— SDN and FPD—were cre<strong>at</strong>ed by<br />
combining previous networks without anticip<strong>at</strong>ing or assessing the impact of<br />
this organiz<strong>at</strong>ional restructuring on the m<strong>at</strong>rix.<br />
Fourth, QAG was dissolved in 2010, placing gre<strong>at</strong>er burden for quality<br />
assurance on the m<strong>at</strong>rix system.<br />
1.46 Isol<strong>at</strong>ing the effects of the m<strong>at</strong>rix system from these overlapping process<br />
reforms is not feasible. While focusing primarily on m<strong>at</strong>rix issues, the evalu<strong>at</strong>ion has<br />
16