The Matrix System at Work - Independent Evaluation Group - World ...

More documents

Recommendations

Info

CHAPTER 4 DELIVERING QUALITY SERVICES 4.22 Regional functions units have grown substantially as they have strengthened their fiduciary and safeguards teams. For example, fiduciary staff in the Regions grew from 53 in FY02 to 407 in FY10. The growth during the first two years can be attributed to remapping of country unit staff to OS units reaching 158 staff in FY05 but the subsequent increase reflects heightened attention to fiduciary areas. A few highprofile investigations of corruption and safeguards have resulted in widespread defensive behavior, leading many country directors and some sector units to add staff to undertake additional quality reviews focused on operational and reputational issues. 4.23 Feedback from staff and managers confirms that the Bank’s quality control systems are more concerned with reputational risks than other aspects of quality. Staff survey results reinforce the findings from managers’ interviews that the quality assurance system is imbalanced. Based on their operational experience, the majority of the 342 staff responding to the survey reported that the quality assurance system ensures high quality for fiduciary aspects (59 percent) and safeguards aspects (54 percent), while one-third report effective use of peer reviews (34 percent) and quality enhancement reviews (33 percent), and high quality of technical design (36 percent). Roughly one-fifth reported that the quality assurance system provided independent feedback on quality, and that there is effective supervision for timely and frank feedback on projects. 4.24 Current incentives and accountability mechanisms lead to task team leaders having the greatest accountability for lending operations and sector directors having the least accountability. Sector directors were identified as primarily accountable for lending or nonlending activities beyond business development in only one of the four Regions that have formal accountability matrices (see Appendix G), while two Regions do not even have formal accountability matrices. Country directors were accountable for concept review meetings, and in all but East Asia and the Pacific, decision meetings. 4.25 Sector managers and country directors were identified as the managers most accountable for lending operations. A third of the 342 respondents to the staff survey reported that sector managers and country directors were accountable for lending quality to a substantial or very large extent, another third considering them accountable to some extent. Their accountability is reported to be significantly lower for AAA quality. At the same time, the span of control of regional sector managers—managing 37 staff on average, compared to seven to 15 in comparator external organizations and 19 in the International Finance Corporation (IFC)—is unrealistic, making it impossible for them to provide quality control for all lending and nonlending operations produced in their unit. They are 69
CHAPTER 4 DELIVERING QUALITY SERVICES instead forced to rely on sub-regional or country-based “shadow managers,” introducing an additional layer of non-technical review, as they are not always experts within the sector. 4.26 Information from managers’ interviews validates these findings. Sector managers typically described accountability for quality as high for the sector manager and the task team leader, none for the sector director, and substantial for the country director. Across all the Regions, sector directors’ accountability for lending operations was rated as substantial (or very large) by only 13 percent (four of 30) of sector managers and 9 percent (one of 11) of country directors interviewed. Sector directors are deemed accountable only for cases where technical assistance has substantial potential impact on country dialogue. In comparison, sector managers are reported to have much greater accountability for lending, which was rated substantial by 90 percent of the sector managers and 55 percent of country directors interviewed. The staff survey yielded similar results. 4.27 Quality assurance systems are described as being clearer and more effective for lending than for ESW, while accountability for NLTA is even weaker. Review processes have been in place for lending operations since before the matrix. A twostage review process was introduced for ESW following the 2001 evaluation of the Strategic Compact, but until FY11 no Bank-wide requirement was in place for review of NLTA quality. Managers and staff report that performance of the current quality assurance system is relatively effective in ensuring fiduciary and safeguards quality, but less so for assuring the quality of lending operations, even lower for the quality of ESW, and lowest for NLTA. Feedback from country directors and sector managers is fairly consistent. Managers find that the current incentive structure results in attention to processes and reputational risk more than operational and AAA quality. Many argued that, as one respondent put it, “too much attention on safeguards and fiduciary takes away from time spent ensuring technical quality of lending operations,” and that “the quality assurance system, such as it is, provides some assurance for fiduciary/safeguard issues and ESW reports, but slight [assurance] for quality of lending and none for NLTA.” The implications of weak accountability of NLTA are especially serious for institutional interventions, such as those in the public sector governance portfolio. 4.28 There is little opportunity for the quality assurance system to materially improve many pieces of ESW. For nearly 40 percent of ESW that have decision meetings—in which a study is put up for institutional scrutiny, and is commented on by peer reviewers—those meetings occur less than 30 days before delivery to the client, as shown in Figure 4.3. It is implausible that substantive peer review critiques of a sophisticated piece of analytical work can be adequately addressed by such a 70
Page 1 and 2:
The Matrix System at Work An Evalua
Page 3 and 4:
©2012 Independent Evaluation Group
Page 5 and 6:
Costs of Country and Sector Strateg
Page 7 and 8:
Table 3.6. Total Cross-Support Sold
Page 10:
Abbreviations AAA AFR ARDE ARPP BET
Page 14 and 15:
Glossary Term Anchor sector directo
Page 16 and 17:
GLOSSARY Practice managers (PM) Reg
Page 18 and 19:
Overview The Matrix System at Work
Page 20 and 21:
OVERVIEW rather than on the 1997 ma
Page 22 and 23:
OVERVIEW satisfactory or better in
Page 24 and 25:
OVERVIEW toward capacity building a
Page 26 and 27:
OVERVIEW Innovation in the field is
Page 28 and 29:
OVERVIEW Figure D. Outcomes of Bank
Page 30 and 31:
OVERVIEW modernization in the corpo
Page 32 and 33:
OVERVIEW management structure but t
Page 34 and 35:
OVERVIEW Structure Bank-wide matrix
Page 36 and 37:
MANAGEMENT RESPONSE interest and ow
Page 38 and 39:
MANAGEMENT RESPONSE Management cons
Page 40:
MANAGEMENT RESPONSE that reach far
Page 43 and 44:
CHAPTER 1 INTRODUCTION AND CONTEXT
Page 45 and 46:
Page 47 and 48:
Page 49 and 50:
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
Page 59 and 60: 2. The Strategic Alignment of Secto
Page 61 and 62: CHAPTER 2 THE STRATEGIC ALIGNMENT O
Page 79 and 80: CHAPTER 3 THE PROMISE OF A KNOWLEDG
Page 103 and 104: CHAPTER 4 DELIVERING QUALITY SERVIC
Page 109: CHAPTER 4 DELIVERING QUALITY SERVIC
Page 123 and 124: CHAPTER 5 INSTITUTIONAL ISSUES AND
Page 145 and 146: 6. Conclusions and Recommendations
Page 147 and 148: CHAPTER 6 CONCLUSIONS AND RECOMMEND
Page 149 and 150: CHAPTER 6 CONCLUSIONS AND RECOMMEND
Page 152 and 153: Appendix A Matrix Management and Or
Page 154 and 155: APPENDIX A MATRIX MANAGEMENT AND OR
Page 160 and 161:
APPENDIX A MATRIX MANAGEMENT AND OR
Page 162 and 163:
SECONDARY ANALYSIS OF CASCR REVIEWS
Page 164 and 165:
APPENDIX B EVALUATION METHODOLOGY a
Page 166 and 167:
APPENDIX C RESPONSIVENESS AND EFFEC
Page 168 and 169:
Page 170 and 171:
Page 172 and 173:
Page 174 and 175:
Page 176 and 177:
APPENDIX D EXPERIENCE IN THE TRENCH
Page 178 and 179:
Page 180 and 181:
Page 182 and 183:
Page 184 and 185:
Page 186 and 187:
Page 188 and 189:
Page 190 and 191:
Page 192 and 193:
Page 194 and 195:
Page 196 and 197:
Appendix E Key Findings from Sector
Page 198 and 199:
APPENDIX E KEY FINDINGS FROM SECTOR
Page 200 and 201:
Page 202 and 203:
Page 204 and 205:
Page 206 and 207:
Table E.1. Country Directors and Ma
Page 208 and 209:
Page 210 and 211:
# Other Questions Q # 7 APPENDIX E
Page 212 and 213:
Page 214 and 215:
APPENDIX F FOCUS GROUP DISCUSSIONS:
Page 216 and 217:
APPENDIX F FOCUS GROUP DISCUSSIONS:
Page 218 and 219:
Appendix G Comparison of the Six Re
Page 220 and 221:
APPENDIX G PROFILES OF THE BANK’S
Page 222 and 223:
Page 224 and 225:
Page 226 and 227:
Appendix H Working across Organizat
Page 228 and 229:
APPENDIX H WORKING ACROSS ORGANIZAT
Page 230 and 231:
Page 232 and 233:
Page 234 and 235:
Page 236 and 237:
Page 238 and 239:
Appendix I Budgetary Incentives 65.
Page 240 and 241:
APPENDIX I BUDGETARY INCENTIVES Bud
Page 242 and 243:
APPENDIX I BUDGETARY INCENTIVES 72.
Page 244 and 245:
APPENDIX I BUDGETARY INCENTIVES com
Page 246 and 247:
APPENDIX I BUDGETARY INCENTIVES Fig
Page 248 and 249:
APPENDIX I BUDGETARY INCENTIVES mea
Page 250 and 251:
APPENDIX J WORKING ACROSS SECTORS A
Page 252 and 253:
Page 254 and 255:
Page 256 and 257:
Appendix K World Bank Sector Strate
Page 258 and 259:
APPENDIX K WORLD BANK SECTOR STRATE
Page 260 and 261:
APPENDIX K WORLD BANK SECTOR STRATE
Page 262 and 263:
Appendix L International Finance Co
Page 264 and 265:
APPENDIX L IFC 2013 CHANGE INITIATI
Page 266 and 267:
Page 268 and 269:
Page 270 and 271:
APPENDIX M STAFFING DATA Note: Head
Page 272 and 273:
APPENDIX M STAFFING DATA Table M.7.
Page 274 and 275:
Appendix N Managers Interviewed Nam
Page 276 and 277:
References Barnett, Michael N. and
Page 278 and 279:
REFERENCES Sy, Thomas and Laura Sue
Page 280 and 281:
Endnotes 1 See Wapenhans 1992, 1994
Page 282 and 283:
ENDNOTES appropriately assessed by
Page 284 and 285:
ENDNOTES 46 The respondents to this
Page 286:
ENDNOTES 71 See Safeguards and Sust
show all

The Matrix System at Work - Independent Evaluation Group - World ...

Create successful ePaper yourself

Delete template?

Save as template?