The Matrix System at Work - Independent Evaluation Group - World ...

OVERVIEW
The Bank’s role in development
knowledge is equally vital to the quality of
its lending portfolio. In addition to
knowledge services that have a direct bearing
on policy advice and institutional development,
there has always been a complementarity
between knowledge and lending. IEG’s
evaluation of the Bank’s use of knowledge to
improve development effectiveness found that
economic and sector work (ESW) and
technical assistance was effective in shaping
Bank lending and was statistically associated
with better loan design (IEG 2008a). The
Bank’s ability to produce high-quality
knowledge needs to be preserved to improve
the quality of Bank lending, and its ability to
access and customize cutting-edge global
knowledge is essential to maintain its relevance
to the increasingly complex demands of its
client countries.
Delivering Quality Services
The matrix system was designed to
strengthen the Bank’s ability to deliver
quality services that combine relevant
local solutions with global knowledge.
This was to be achieved by the combination
of country priorities articulated by country
management and global technical expertise
brought to bear by sector management and
the Bank-wide networks.
Outcomes of Bank-financed operations as
measured by year of exit from the
portfolio improved although the system is
under increasing stress in recent years.
The aggregate share of operations rated
moderately satisfactory or better at exit after
FY97 is significantly higher than before the
matrix was introduced. But outcome ratings
by exit year have declined from a peak of 79
percent moderately satisfactory or better in
FY05-07 to 74 percent in FY08-10 (Figure D),
which is statistically significant. Reporting
outcomes by year of exit is a lagging indicator
of quality at entry, reflecting performance of
projects prepared an average of seven years
earlier and affected in part by Bank
supervision.
When project outcomes are compared by
year of approval, the most significant
improvement in the share of Bankfinanced
operations with moderately
satisfactory or better outcomes occurred
before the introduction of the matrix, and
a downward trend is evident after a peak
in FY01 (Figure E). Outcome ratings by year
of entry show that the most significant
increase occurred in projects approved before
the introduction of the matrix. The slight gain
for projects approved during the early years of
the matrix (FY1998-01) was lost by projects
approved after FY01. Using a three-year
moving average, the share of operations rated
moderately satisfactory or better among those
approved during FY01-03 was 79 percent,
compared to 70 percent among operations
approved during FY04-05, a drop that is
statistically significant with 95 percent
confidence.
The quality of supervision and quality at
entry have shown a declining trend since
the introduction of the matrix. In addition
to assessing the outcomes of operations after
they have closed, IEG assesses Bank
performance in ensuring quality at entry and
the quality of supervision. Data from the
portfolio of investment projects from
FY1990-2010 reveal that quality at entry
peaked in FY96-98, while quality of
supervision peaked in FY97-99. Portfolio
indicators of supervision quality tracked by
management also show a decline in Bank
performance, despite recent increases in
supervision budgets. These may be
interpreted as leading indicators of portfolio
quality.
Multiplication of quality control layers for
lending has increased neither efficiency
nor effectiveness in ensuring quality.
Accountability for quality has become more
diffused with more cumbersome review
xxiv