04.04.2017 Views

eVALUation Matters

1gGWOV

1gGWOV

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

Fourth Quarter 2016

eVALUation Matters

A Quarterly Knowledge Publication on Development Evaluation

Evaluation Week 2016


Published by

eVALUation Matters

is a quarterly publication from Independent

Development Evaluation at the African Development

Bank Group. It provides different perspectives and

insights on evaluation and development issues.

Independent Development Evaluation

African Development Bank

from experience to knowledge…

from knowledge to action…

from action to impact

Editor-in-Chief: Kate Stoney, Communication

Consultant

Acknowledgments: IDEV is grateful to all contributors,

reviewers, editors, and proofreaders who worked on this

issue, in particular:

• AfDB Language Services Department, French

translation

• Deborah Glassman, editor

• Karen Rot-Munstermann, Manager, IDEV 3

Design: Visual Identity

Layout: Visual Identity

About Independent Development Evaluation

The mission of Independent Development Evaluation at

the AfDB is to enhance the development effectiveness

of the institution in its regional member countries

through independent and instrumental evaluations and

partnerships for sharing knowledge.

Evaluator General: Rakesh Nangia, r.nangia@afdb.org

Managers:

Rafika Amira, r.amira@afdb.org

Samer Hachem, s.hachem@afdb.org

Karen Rot-Munstermann, k.rot@afdb.org

Questions?

Phone (IDEV): +225 2026 2041

Phone (AfDB standard): +225 20 26 44 44

Write to us:

email: idevhelpdesk@afdb.org

Mail: 01 BP 1387

Abidjan 01

Côte d’Ivoire

Web:

IDEV: idev.afdb.org

AfDB: afdb.org

Copyright: © 2016 – African Development Bank (AfDB)


To accompany the AfDB’s new development

strategy to transform Africa, the Independent

Development Evaluation (IDEV) of the Bank

held its Development Evaluation Week 2016 on

the theme “Driving Africa’s Transformation”.

Evaluation experts, academics and practitioners from

diverse horizons converged in Abidjan to examine

the Bank’s High-5s: Light up and power Africa;

Feed Africa; Integrate Africa; Industrialize Africa; and

Improve the quality of life for Africans. Participants

at Evaluation Week discussed how lessons learned

from the AfDB and other development institutions

can inform the design and implementation of policies

and operations in the five priority areas, and how

innovative solutions can contribute to success.

How should we go about evaluating the High 5s? What

kind of theory of change will guide IDEV’s evaluation

in such a context? Will IDEV continue to rely on its

conventional evaluation criteria or will it fashion new

ones that will focus on the specificities of the High 5s?


From left to right: Aka Hyacinthe Kouassi,

Koffi Yao, Nialé Kaba, Rakesh Nangia

iv

eVALUation Matters / Second quarter 2016


“Evaluation is one of our weaknesses.

We develop policies, but

in the end, we do not check if we

have reached the expected results.’’

Koffi Yao, Côte d’Ivoire Ministry of Planning and Development

v

Independent Development Evaluation


Contents

Page

“The money is there, but

there are no bankable

projects. The Bank needs

to play a catalytic role to

crowd-in more private

sector investments in the

energy market in Africa”

Amadou Hott

From the Evaluator General’s Desk

Achieving transformation requires not only investments and policies, but also a change

in mindset. With this in mind, we at IDEV organized the AfDB Development Evaluation

Week 2016 on the theme of Driving Africa’s Transformation. The number of panel

discussions that overran into coffee and lunch breaks are a good indicator of the symbiosis

formed between panellists and with the audience, and the enthusiasm that the

discussions at Evaluation Week sparked.

Rakesh Nangia, African Development Bank

The Knowledge Café in pictures

The Evaluation Week’s Knowledge Café was an opportunity to brainstorm and put

forward candid opinions on how to better use evaluation to achieve the High-5s. Talking

points on each of the themes are collated and presented in Knowledge Café- branded

sections of this magazine.

Light Up and Power Africa

“Lighting up and Powering Africa – on the road to renewable energies” begs panellists

to share their knowledge and experience of using innovative solutions. Issues ranging

from capacity imperatives, governance issues, financial, operational, technological,

regulatory support, and leadership are raised during the session.

Ronald Meyer, Executive Director, AfDB; Marc Albérola, CEO, Eranove Group/

Director of Operations, Eranove Cote d’Ivoire; Elias Ayuk, Director, UNU-INRA,

Accra; Amadou Hott, Vice President, Power, Energy, Climate and Green Growth,

AfDB; Aka Hyacinthe Kouassi, Senior Advisor, AfDB; Vanessa Ushie, Pan-Africa

Senior Policy Advisor on Extractive Industries, Oxfam.

Feed Africa

The panellists discuss the fundamental questions of who should feed Africa and how

should Africa be fed, giving their viewpoint on the commitment of African governments

and institutions to address the drivers of hunger, poor nutritional outcomes, and the

issue of food security.

J. Chris Toe, Senior Advisor for Country Strategic Planning, World Food

Programme; Anne-Sophie Le Dain, Nutrition Manager, UNICEF Côte d’Ivoire;

George Mavratos, Senior Research Fellow & Program Lead, Nigeria Strategy

Support Program, IFPRI; Fadel Ndiame, Head of AGRA West Africa operations;

Oscar Garcia, Director, Independent Office of Evaluation, IFAD; Daniel Alberts,

Senior Manager, GAIN; Chiji Ojukwu, Director, Agriculture and Agro-Industry

Department, AfDB.

Leadership Matters

Tuesday’s high-level panel gave food for thought to political leaders. West African

ex-Ministers expressed the necessity to improve the leadership capacity in our African

states, at all levels and in all areas.

Kako Nubukpo, Economic and Digital Director, IOF, Former Minister of Togo;

Antonin Dossou, Director at the BCEAO, Former Minister of Benin.

Industrialize Africa

This article features a presentation of findings from a Private Sector Evaluation synthesis,

the viewpoints of panellists on the way governments could adopt policies that would

propel sustainable industrialization, and the means to establish effective cooperation

between the government and the private sector. The session is illustrated with case

studies of African industrialization.

Sumir Lal, Director, External Affairs, World Bank; Massogbè Toure-Diabate,

Chairperson, Women Entrepreneurship Committee, Confederation of Large

Businesses in Côte d’Ivoire (CGECI); Per Øyvind Bastøe, Director, Evaluation

Department, Norad; Mariam Dao Gabala, CEO of MDG Consulting and Chair

of the Board of Solidaridad; Soraya Mellali, Executive Director, AfDB; Edward

1

4

10

18

31

32


Page

Marlow, Head of Sub Saharan Africa Client Group, Credit Suisse; Tim Turner,

Chief Risk Officer, AfDB.

Integrate Africa

Experts put forward that the need to get politics and governance right is even more

urgent than economics and funding of regional integration. How can we define and

demonstrate the impact of regional integration to stimulate stakeholders’ interest in

it? How can we accommodate political, economic and cultural differences in regional

projects to ensure the buy-in of all stakeholders?

40

Fredrik Söderbaum, University of Gothenburg, Sweden; Nyamajeje Calleb

Weggoro, Executive Director, AfDB.

Evaluation Choices

44

An evaluation expert summarizes the roles and outreach of an evaluation department

within an organisation.

Ruben Lamdany, Deputy Director, Independent Evaluation Office of the International

Monetary Fund.

Improve the quality of life for the people of Africa

46

The main challenge for the success of the AfDB High 5s, beyond the quality of program

and project design, is primarily political and sociological, claims a thought leader. Meanwhile,

on the ground, mobile technology can help communities improve their quality of

life in practical ways. This article presents views from both sides of the fence.

Antonin Dossou, Director at the BCEAO, Former Minister of Benin; Charles

Boamah, VP Finance at AfDB; Batio Bassière, Minister of the Environment,

Burkina Faso; Pindai M. Sithole, Centre for Development in Research and Evaluation

International Africa (CeDRE Africa); Miltiade Tchifou Dieffi, Cameroonian

association for the development of evaluation.

The Evaluation Week contests

Photos of the winners of the Evaluation Week essay and photo contests.

Impact Evaluation in International Development

What is, and what should be the contribution of Impact Evaluation to inform policy

dialogue? Panellists discuss the role and importance of Impact Evaluation against a

backdrop of challenges in methodologies and high associated costs.

Abebe Shimeles, Acting director of the Research Department (EDRE), AfDB;

Patrice Bosso, UNICEF, Côte d’Ivoire; Zenda Ofir, President ICED; Maria Sophia

Aguirre, Catholic University of America.

54

58

Evaluation Matters: Value for money in development work

eVALUation Matters

A Quarterly Knowledge Publication on Development Evaluation

Third Quarter 2016

Climbing the High 5 learning curve with independent evaluation

Evaluation experts discuss the current trends in evaluation and how we should further

adapt methodologies to evaluate the AfDB High 5s. The panellists share their experience,

including a section on lessons learned by the UNDP, and give thoughtful advice

to evaluators on how to take the profession forward.

Simon Mizrahi, Director, Results & Quality Assurance Department, AfDB; Zenda

Ofir, President ICED; Keith Leonard, Development Evaluation Expert, EBRD;

Indran Naidoo, Director Evaluation Office, UNDP; Rakesh Nangia, Evaluator

General, AfDB; Oscar Garcia, Director, Independent Office of Evaluation, IFAD;

Saphira Patel, Evaluation Department DBSA.

Evaluation Week program

The full schedule of events day by day, with the presenters and panellists.

64

72

Value for money

in development work

Check out Evaluation

Matters third quarter:

http://idev.afdb.org/en/document/

evaluation-matters-third-quarter-2016-

value-money-development


viii

eVALUation Matters / Fourth quarter 2016


Foreword by the Evaluator General

The African Development Bank’s Ten-Year

Strategy places the Bank “at the heart

of Africa’s transformation”. Achieving

transformation requires not only investments

and policies, but also a change in

mindset that draws on what works, what

does not work – and why – to succeed.

Transformation also calls for enhancing

requisite capacities – both individual

and institutional-, an essential aspect in

today’s fast changing and very competitive

world, and central to delivering on

well-intentioned policies. In essence,

Africa needs to learn from experience

and results.

With this in mind, we at IDEV organized

the AfDB Development Evaluation Week

2016 on the theme of Driving Africa’s

Transformation. We decided to open the

debate on evaluating this transformation

through the High-5s: Light up and power

Africa, Feed Africa, Industrialize Africa,

Integrate Africa, and Improve the quality

of life for the people of Africa, and

how evaluative knowledge can help the

Bank improve the implementation of the

High 5s.

The premise was that ‘good-fit’

approaches to evaluating complex initiatives

taken to induce the continent’s

transformation would ensue from the

intertwining of viewpoints and positions,

experiences and skill-sets. Hence,

we convened fellow development stakeholders

to share evidence-based knowledge

and expertise from their countries

and backgrounds. The number of panel

discussions that overran into coffee and

lunch breaks are a good indicator of the

You must learn to make the

whole world your school.

Martin Fischer

symbiosis formed between panelists and

with the audience, and the enthusiasm

that the discussion sparked.

On a personal note, it was heart-warming

for me to meet up again with fellow evaluation

experts who have over the years

become “friends in development”, and an

honor for me to have an armchair “one to

one” discussion with our Senior V.P. Frannie

Léautier on how we can improve the

quality of life for the people of Africa.

The dialogue forum attracted over 250

participants – from MDBs, the private

sector, public institutions, policy makers,

academics, evaluators, development

practitioners, students, media, etc.

Presentations and panel discussions,

grounded in evaluative evidence, and

enhanced by invaluable input from the

audience, focused the dialogue while

privileging the voices of all.

Questions debated included: What

evidence exists on what works, what does

not work, and why? What lessons emanating

from AfDB and partner interventions

can inform the design and implementation

of policies and operations in the five

priority areas? What innovative solutions

out there can inform the success of

planned initiatives? What should success

look like, and how should we measure

and evaluate success/failure? How

1

Evaluation Week 2016


Frannie Léautier, speaks to Rakesh Nangia.

“There is a very tight link between evaluation and

innovation. Innovations that succeed are those

where you have tried, failed, recovered, re-structured,

learned from that and moved on. As many innovations

fail, it is important to draw lessons from those

failures, which in turn can fuel further innovations.”

Frannie Léautier

appropriate, effective, and sustainable

are AfDB’s envisioned monitoring and

evaluation strategies/measures for the

High-5s? How can we put more emphasis

on questions that matter to society such

as inclusion, gender, equity, efficiency

and sustainability?

While focused on evidence and results,

the event made room for light moments

too. Monday’s very interactive and iterative

Knowledge Café and Evaluation

Capacity Development Workshop helped

participants break the ice and get to

know each other around the discussion

table. On Tuesday, I was assigned the role

of “father” of IDEV at the Akwaba Ceremony.

In Côte d’Ivoire, Akwaba means

“welcome” and I therefore welcomed

“guests from all the corners of the globe”

in a traditional way; though I was relieved

that the IDEV team had organized professional

entertainers to round up the ceremony!

The Akwaba ceremony was indeed

an opportunity for guests to admire the

precision of traditional dancers’ footwork

and contortions, and drummers’ rhythmic

flows.

This edition of eVALUation Matters is

dedicated to Development Evaluation

Week 2016, and synthesizes viewpoints

and reflections on evaluative knowledge

expressed during the week. Editorial

contributions are compiled from eminent

keynote speeches and recordings of

2

eVALUation Matters / Fourth quarter 2016


panelist discussions, verbatim from the

Knowledge Café, Evaluation Week award

winners and lunch-time presenters. Each,

in their own way, provides a nuanced and

granular understanding of contemporary

Africa and the evaluation landscape.

Looking ahead, I am confident that we

will build on the momentum of Evaluation

Week and be bold in forging ahead;

that countries will not only enhance their

capacity choices but also their policy

options; and that we will collectively work

at using evaluation lessons to eliminate

the bottlenecks and meet the targets

of the High-5s for the benefit of all the

people of Africa.

Author’s Profile

Rakesh Nangia is the Evaluator General at the African Development

Bank. Prior to joining the AfDB, he spent 25 years at the

World Bank where he held several positions including Director

of Strategy and Operations for the Human Development

Network and Acting Vice-President for the World Bank Institute.

He attended the Indian Institute of Technology in Delhi and

Harvard University and holds degrees in business administration

and engineering.

Evaluation Week 2016

3


https://goo.gl/DemhIO

The

Knowledge

Café concept

IDEV Evaluator, Foday Turay, introducing the Knowledge Café.

4

eVALUation Matters / Fourth quarter 2016


A knowledge café is a means of bringing

a group of people together to have

an open, creative conversation on a

topic of mutual interest.

The IDEV knowledge café was set up in

the lobby at AfDB Headquarters in Abidjan.

Five thematic tables (one for each

of the AfDB High 5s) were set up in ‘café’

style to create a relaxed, informal ambience.

The facilitator, IDEV Evaluator

Foday Turay, introduced the knowledge

café and invited participants to choose a

table to start the process.

The Knowledge Mentor on each table

then gave brief guidelines, prompting

the participants in their discussion.

After 20 minutes, the participants

rotated to the next table, while the host

stayed behind to introduce the question

once more and summarize the preceding

discussion for a new set of participants.

The new participants then add

their insights to the question, refining

or modifying the contributions of the

previous group. A concluding plenary

wrapped up the discussions.

“The Knowledge Café was an opportunity to brainstorm

and put forward candid opinions on how to better

use evaluation to achieve the High 5s. On my table, it

was put forward that beneficiaries of a development

programme would be more inclined to take note of

evaluation lessons learned if they were involved in

the programme from the design phase. Perhaps this

is one way to gain in sustainable development”

Amadou Boly, participant at the “Quality of Life” table.

Evaluation Week 2016

5


The Knowledge Café Concept

The Knowledge Café tables

Industrialize Africa

Feed Africa

6

eVALUation Matters / Fourth quarter 2016


Light up and Power Africa

Integrate Africa

7

Evaluation Week 2016


Feed Africa

8

eVALUation Matters / Fourth quarter 2016


Delivering a Keynote on behalf of the

Prime Minister of Côte d’Ivoire, Nialé Kaba,

Minister of Planning and Development of

Côte d’Ivoire, stressed the importance of

evaluating the High-5s, which collectively are

fundamental to transforming the continent.

9

Evaluation Week 2016


https://goo.gl/L9VrFY

Light Up and

Power Africa

David Mbuthia Mwangi,

Freelancer, Kenya

10

Evaluation Week Photo Contest – Third prize

Apodo Alphonce is a young Kenyan. A product

of his generation, he uses a solar-powered lamp

to read and light his home. Solar energy is now a

relatively cheap and reliable source of power for

millions of Kenyans.

eVALUation Matters / Fourth quarter 2016


From left to right: Elias Ayuk, Director, UNU-INRA, Accra; Aka Hyacinthe

Kouassi, Senior Advisor, AfDB; Vanessa Ushie, Pan-Africa Senior Policy

Advisor on Extractive Industries, Oxfam; Moderator, Ronald Meyer, Executive

Direcor, AfDB; Marc Albérola, CEO, Eranove Group/Director of Operations,

Eranove Côte d’Ivoire; Amadou Hott, Vice-President, Power, Energy, Climate and

Green Growth, AfDB.

Amadou Hott

“The money is there, but

there are no bankable

projects. The Bank needs

to play a catalytic role to

crowd-in more private

sector investments in the

energy market in Africa”

Aiming to resolve our electricity problems

within the next ten years is a daunting

task, but possible if we build strong

partnerships. Vietnam and Bangladesh

have achieved extraordinary results

over the last years, and Kenya is our

continent’s showcase country. Donor

money alone will not be sufficient; we

need $40-50 billion/year over the next

ten years to achieve our renewable

energy targets. The Bank must play a

leadership role to coordinate the various

efforts, and to leverage development

finance institution funds to catalyze

private sector money in the form of

pension funds, private equity funds, etc.

PPP is a viable option to solve “bankable”

issues but institutional capacity at

the government side remains an issue

for investors.

11

Evaluation Week 2016


Light Up and Power Africa

“We need a regulatory framework

to favor off-grid energy.”

Marc Albérola

We are obviously interested in developing

renewable energies, which are abundant,

locally available and therefore offer

options for competitively priced energy

access. Governments and their development

partners must find the institutional

and regulatory arrangements

that favor this type of energy development.

In Mali, which has the capacity

and investment framework, we are

currently building a small 42 MW hydro

power plant on the Niger river.

We believe that it is preferable to develop

very local, off-grid renewable energy

solutions rather than investing in large

power plants to produce renewable energies,

as these create other issues such as

costly distribution networks.

Government focus should be on accompanying

local micro-hydraulic and biomass

power-production projects, especially in

areas without electricity. Local companies

and individuals should be capacitated

to exploit them to an extent where they

will develop revenue-generating activities

and therefore the capacity to auto-finance

solutions and pay for the services.

PPPs are vital elements in offering affordable

energy solutions. To bring down the

price of services to an acceptable level

for rural populations where the private

sector finances let’s say 30% of a power

plant project, the State could contract the

remaining 70% of financing, passing on

the benefit of its loan conditions to end

users of the energy services.

12

eVALUation Matters / Fourth quarter 2016


“We need incentives to drive

power supply away from

fossil fuels towards a more

diversified mix of energy.”

Vanessa Ushie

When discussing powering Africa and

closing the energy gap, we must distinguish

between interrelated structural

issues. Is our focus on providing energy

for industrial use or giving access to

energy to the poor? The latter is just as

strategic, since it equates to meeting the

demands of the poor for multiple uses

-cooking, education, small-scale agriculture

and small businesses.

The heavy carbon footprint of natural

resource extraction is highly regressive,

creates few local jobs, and diverts

resources. Climate change is affecting

how we transform our economies, societies,

and lives in Africa. The New Deal and

High 5s in general must build local resilience

and ensure that projects designed

to increase energy access are climateproof

investments. Regional development

financial institutions must confront the

structural problem of relying on natural

resource extraction as a basis for industrialization.

In the Bank’s shift to renewable

energy, it must find the right balance

between resources and incentives, and

between off-grid and on-grid electricity.

Falling prices of renewables, growing

numbers of rooftop smart and

micro-grids, improvements in power

efficiency, reduction in waste and

emissions; all need to be tracked

regularly. Environmental and health

outcomes should be evaluated in

parallel. What are the outcomes

for women’s health, for example, of

the clean-cook stove initiative?

13

Evaluation Week 2016


Light Up and Power Africa

“In the current energy mix,

Africa relies on fossil fuels for

50% of its energy, which is

unacceptable.”

Aka Hyacinthe Kouassi

In the current energy mix, Africa relies

on fossil fuels for 50% of its energy,

which is unacceptable. This must and

can change given Africa’s enormous

production potential using renewable

resources. Côte d’Ivoire, Guinea, and

Equatorial Guinea are examples of countries

that produce electricity primarily

using hydro and thermal energy.

We need to strengthen governance in

the energy sector, carry out an evalua-

tion of the renewable energy potential,

and draw up a clear policy promoting

renewable energy and ending subsidies

to the fossil fuel industry. The institutions

created to promote renewable

energies remain undeveloped.

It is important to establish a regulatory

framework that encourages private

investments, with appropriate implementation

measures and reassurance

of government commitment.

The role of a Development Finance Institution is to:

• Build government capacity.

• Help governments develop a diversified energy mix.

• Increase the number of bankable renewable energy projects.

• Provide the financial resources necessary to produce a leverage effect for the

market and trigger private sector investments.

• Increase programs for small-scale financing in rural areas, in partnership with

social funds, NGOs, and other development initiatives.

14

eVALUation Matters / Fourth quarter 2016


“In Ghana, we lose 12% of our

GDP to environmental degradation.

We cannot grow now,

then debate and clean up later.”

Elias Ayuk

We know that when the space shuttle

passes over Africa, it is dark. We have

done too little to provide electricity in a

context where energy holds the key to

structural transformation in many countries.

When evaluating the benefits of

providing power, we should take a holistic

approach that encompasses knock-on

effects to education, health, and small

and medium-sized enterprises etc.

In Ghana, we have been challenged to

provide sufficient power over the last few

years, and this has had direct implications

for SMEs. The power supply is therefore

an urgent issue for the country’s development

and we must include renewable

energy alternatives when we address it.

The USA has a deliberate strategy to

pursue fossil fuels. Next year it will be

the world’s largest oil producer. Here in

Africa, the cost of environmental degradation

is high and is growing. Energy

access directly impacts health, education,

and trends in rural-urban migration. 2012

data shows that in Ghana we lose 12% of

our GDP to environmental degradation.

We cannot grow now and then debate

and clean up later.

I’d like to suggest that we follow the six I’s

to achieve our aspirations: Ideas, Incentives,

Innovations, Institutions – political

institutions, market institutions, etc.,

Infrastructure, and Implementation.

15

Evaluation Week 2016


Light Up and Power Africa

Batio Bassière, Minister of the Environment,

Burkina Faso

“We need to promote a green economy

through regulations that foster

investment, effectively managing

technology transfer, and by providing

innovative financing mechanisms

for the private sector that break de

facto monopolies. In Burkina Faso,

a National Renewable Energies

Promotion Agency was recently

set up at a Council of Ministers’

meeting. A solar power station is

under construction in Zagtouli to

supply energy to Ouagadougou.”

“15 years ago, a country-led program in Rwanda aimed to put

up off-grid power facilities in every community across the

country. The government remained committed and involved

throughout, and the implementation has been successful”

Ramachandra Jammi, Senior Evaluator, IEG, World Bank.

16

eVALUation Matters / Fourth quarter 2016


Knowledge Café

What are the recommendations for an

effective implementation of this High 5?

Participants noted that with its ‘New

Deal on Energy’, the Bank is now adopting

a more holistic approach to light up

and power Africa.

Participants from IDEV, currently

conducting an evaluation of AfDB funded

rural electrification projects, point out

that less than 5 percent of rural populations

in sub-Saharan Africa have access

to electricity, the lowest access rate in the

world. This is due to the fact that the high

cost of electricity supply in rural areas

and the limited capacity of households to

pay for services make it difficult to attract

investment. At country level, there should

be a system of tariffs and subsidies that

ensures sustainable cost recovery while

minimizing price distortions. In most

countries, subsidies have so far failed to

provide utilities with incentives to invest

in rural electrification.

The approach focuses less on projects

and more on solutions to power problems

in Africa. The mandate of the Bank

includes providing contextual advice

which fits the country’s strategy for development.

The Bank is enjoined to first

identify the needs of a country prior to

its intervention there. To facilitate implementation,

participants unanimously

think that the Bank should carry out evaluability

studies prior to its intervention

in the power sector. Finally, the Bank is

urged to draw appropriate lessons from

its own experience prior to any new intervention;

harmonize with good practices

of other donors on the Continent in order

to enhance development effectiveness;

and reinforce the monitoring system as

basis for ensuring success.

17

Evaluation Week 2016


https://goo.gl/ZfG2gX

Feed

Africa

18

eVALUation Matters / Fourth quarter 2016


From left to right: Anne-Sophie Le Dain, Nutrition Manager, UNICEF Côte d’Ivoire,

George Mavratos, Senior Research Fellow & Program Lead, Nigeria Strategy Support

Program, IFPRI; Fadel Ndiame, Head of AGRA West Africa operations, Moderator:

Oscar Garcia, Director, Independent Office of Evaluation, IFAD, Daniel Alberts, Senior

Manager, GAIN, Chiji Ojukwu, Director, Agriculture and Agro-Industry Department, AfDB

Keynote Address

by J. Chris Toe, Senior Advisor for Country Strategic Planning at the World Food Programme

The region remains home to a disproportionate

share of the global burden

of hunger and undernutrition at a time

when the economic tailwinds that helped

many countries reduce extreme poverty

and improve other human development

indicators over the last decade

are waning; and the impediments are

multi-dimensional and complex:

Where do we begin to fulfil our commitment,

obligation and responsibility to the

hundreds of millions for whom despair,

depravity and hopelessness have become

the new normal? Where do we start to

drive the transformative change that has

eluded Africa for far too long, but must

now be delivered in the interest of today

and tomorrow’s generations?

I believe we must start by focusing on the

most vulnerable and the recognition that

reaching Africa’s hungry poor requires a

fundamental revolution in how we collectively

work; and that doing business as

usual is insufficient. If we fail to change

what we do, and how much we do, we will

not realize change in the time or at the

scaled-up levels required.

In fact, if we keep responding in the

same way, evidence suggests that by

2030 some 650 million people, many of

them in Africa, will remain left behind,

trapped in hunger’s vicious cycle.¹ And

if we fail to mitigate climate-change by

2050, an additional 200 million, many

in Africa again, will join the ranks of the

hungry poor.

Achieving food and nutrition security for

those furthest behind, in Africa and in the

world, thus requires addressing several

blind-spots.

¹ Source: FAO, IFAD and WFP (2015). Investments Need for Eliminating Hunger by 2030.

19

Evaluation Week 2016


Feed Africa

The first is to address conflicts, which

are increasingly at the root of protracted

crises, and along with the resulting

instability, the leading causes of food

insecurity and undernutrition. In 1990,

12 countries in Africa were facing food

crises. Just 20 years later, a total of 24

countries in Africa were in food crises,

with 19 of these in protracted crisis and

therefore acutely vulnerable.

More than 125 million people are

currently affected by humanitarian

crises,² including in conflict settings such

as Nigeria, South Sudan and the Central

Africa Republic. Greater effort toward

reconciliation and understanding will be

required, and strong political commitment

is necessary to address the roots of

protracted crises. Action should focus on

addressing vulnerability, respecting basic

human rights and integrating humanitarian

and development assistance.

The second action is to address climate

risks for smallholder farmers, at the frontline

of food security and stability in Africa.

We must prioritize support to smallholder

farmers, especially those who are

impacted by climate change and conflict.

Climate change affects all of the components

of food security: availability, access,

utilization and stability. According to FAO,

climate change could reduce potential

agricultural output by up to 30 percent in

Africa, and cause food prices in Africa to

rise by 12 percent in 2030 and 70 percent

by 2080. Realizing the potential of Africa’s

small farms will require employing

new tools, including mobile technology

for information transfer as well money

management; and expanding access to

data analysis tools that will enable governments

and other organizations to more

effectively support farmer needs and

increase private sector investment.

The third blind spot is the extension

of social protection and safety nets to

address the challenges of the bottom

quintile, and to leave no one behind.

Chris Toe

“Through Sustainable Development

Goal 2, our leaders

have called for accelerated

action to end hunger, achieve

food security and improved

nutrition and promote sustainable

agriculture”

¹ “Commitment to Action – Transcending humanitarian-development divides – Changing People’s Lives:

From Delivering Aid to Ending Need.” World Humanitarian Summit.

20

eVALUation Matters / Fourth quarter 2016


I believe we must start by focusing on the most

vulnerable and the recognition that reaching Africa’s

hungry poor requires a fundamental revolution in

how we collectively work; and that doing business

as usual is insufficient. If we fail to change what we

do, and how much we do, we will not realize change

in the time or at the scaled-up levels required.

Social protection comes in many forms,

including cash transfers, school meals,

public works schemes, and insurance

programmes. In conflict settings,

protracted crises or simply situations

of extreme poverty, including in Africa,

school meals efficiently combat hunger

and reduce absenteeism, and they

provide incentives for children, particularly

girls, to learn and stay in school.

Rolling-out social protection for all the

world’s hungry people, including Africa,

may seem like a pipe-dream; it is not. A

2015 study undertaken together with

FAO and IFAD estimates that an average

annual investment of only $267 million

dollars would be required. This is just

a fraction of hunger’s estimated $3.5

billion yearly cost.

The fourth action is to empower women,

and in so doing reinforce their roles as

the drivers of development and peace.

In Sub-Saharan Africa where women

provide between 40 and 80 percent of

the labour-force on smallholder farms,

only 5 percent of extension resources

are targeted specifically to women; they

own or manage less than a quarter of the

total agricultural land; and they access

less than 10% of all agricultural credit.

The world’s poorest women require

well-designed financial investments

such as credit or program implementation,

including extension schemes to

empower them to transform their lives.

These changes must be accompanied by

land policy reform which gives the most

vulnerable more access to land ownership,

including inheritance opportunities.

The fifth and final blind spot is to harness

the power and embrace the opportunity

for building strategic partnerships to feed

Africa and to achieve zero hunger in our

lifetime. Progress on the two fronts will

depend on partnerships that go beyond

and across sectors and areas of expertise.

Multi-stakeholder engagement is crucial.

The Sustainable Development Goals and

Africa’s continental frameworks are a

once-in-a-generation chance for all of

us –governments, private sector, donors,

national NGOs, international NGOs, civil

society, UN agencies – to work together

in addressing these critical blind spots.

Doing so will ensure that Africans, particularly

the poorest and most vulnerable

people, not only have the opportunity to

feed themselves today and tomorrow, but

that they also have the hope that we will

consign hunger to history.”

21

Evaluation Week 2016


Feed Africa

“The challenge is to ensure

the translation of policies

and strategies into concrete

action at a decentralized level,

with solid monitoring and

evaluation systems, as well as

equitable implementation”

Anne-Sophie Le Dain

A foundation for

sustainable development.

Nutrition is a pillar of equitable and inclusive

economic development. It should

be at the heart of development action

in Africa because the lack of action

has extremely serious consequences

for the survival of the most vulnerable,

contributing largely to child/infant

morbidity/mortality and maternal death,

and growth retardation. The latter is

also associated with non-optimal cerebral

development likely to have lasting

repercussions on intellectual capacity,

academic performance and future

remuneration. In turn, this affects the

economic and social development potential

of countries that leads to loss of 3 to

17% of GDP. Some countries have already

shown considerable progress in reducing

growth retardation – Ethiopia, Ghana,

Kenya, Liberia, Rwanda and Tanzania,

to mention but a few. This is proof that

it is possible to make a difference. It is

urgent that more countries follow this

trend. To do so, evidence shows that it is

necessary to develop national multi-sector

approaches, such as water, hygiene

and sanitation, health, education, early

childhood development, social and agricultural

policies. The prevention of

malnutrition is one of the most cost-effective

interventions to break the poverty

cycle and contribute to the attainment of

the Sustainable Development Goals that

are closely linked to nutrition.

A nutrition-sensitive

agricultural sector

By investing in nutrition-sensitive policies

and programs, the agricultural

sector will become more efficient and

could maximize its impact on the continent’s

human and economic development.

Key actions involve:

a) Emphasis on improving access to

nutritive foods (production, conservation,

processing, transportation),

including for young children. This

requires strengthening partnership

with the private/agrifood sector;

22

eVALUation Matters / Fourth quarter 2016


) Considering the role of women in agriculture,

their empowerment and the

consequences of women’s agricultural

work on childcare when designing

programs;

c) The use of agricultural programmes

as platforms to offer and improve the

demand for direct nutrition interventions,

for example by engaging agricultural

workers in community-based

programs;

d) The contribution of the agricultural

sector to the creation of a critical

mass of human resources engaged in

nutrition: continuous training but also

incorporation of nutrition in the curricula

of agricultural training schools (e.g.

technicians, engineers, etc.).

e) Systematic integration of nutrition

objectives and measurable indicators

in agricultural programs.

A nutrition-sensitive agricultural sector

will contribute to the global drive to

sustainably prevent growth retardation

and all forms of malnutrition, to

ensure the optimal development of African

people and nations. The challenge

is to ensure the translation of policies

and strategies into concrete action at a

decentralized level, with solid monitoring

and evaluation systems, as well as equitable

implementation.

“Do we want an African food

system designed to produce

stunting and malnutrition, as

it appears to be doing today?”

Daniel Alberts

The High 5s translate “Nourrir l’Afrique”

as Feed Africa. Feeding Africa

must also mean nourishing Africa, as

the French term implies. Is the African

food system designed to be nourishing?

If you consistently get the same result,

you know that the system is designed to

give these results. Do we want an African

food system designed to produce stunting

and malnutrition, as it appears to be

doing today?

From 1990 to 2014, the numbers of

stunted children in Africa rose from 47

to 58 million while over the same period

the number of overweight children rose

from 5 million to 10 million. We must

make sure that the food we are producing

and consuming helps people meet

23

Evaluation Week 2016


Feed Africa

From 1990 to 2014, the

numbers of stunted children in

Africa rose from 47 to 58 million

while over the same period the

number of overweight children

rose from 5 million to 10 million.

their micronutrient needs. Micronutrient

deficiency greatly impacts development,

leaving people without sufficient cognitive

capacity or the ability to work. While

the estimates of costs differ, they represent

about 10% of GDP, which is too high.

Some images are characteristic of

nations and disparities in nutritional

education. That of a woman in Malawi,

sitting beside the road under a tree trying

to sell five tomatoes, remaining there

until the tomatoes are sold or spoiled and

fed to animals or eaten by her. That of a

mother in Tanzania who, when she buys

groundnuts or maize, has no idea about

their possible toxins and the government’s

role in protecting her. That of a

mother in Kenya producing or buying

fortified vegetable oil for her family but

unaware that she is helping to improve

the micronutrient status of her family

thanks to the political will of her country.

We see an image of a teenager in Swaziland

buying Coca Cola and a loaf of white

bread for lunch.

Our goal is to identify how to achieve

and measure impact. I suggest that we

consider everything in-between agriculture

and consumption and find ways to

measure the impact we are having on the

system. A lot more work must be done to

understand the food system that we are

working with. We know that you cannot

move what you cannot measure but you

cannot measure a concept that you do

not understand.

The government has a very strong role

to play, as does the private sector and

the development actors. They can all

contribute to establishing a food system

that helps achieve economic development

from agriculture and that meets

the micronutrient and nutrition needs.

24

eVALUation Matters / Fourth quarter 2016


“We must ensure that there

are incentives for young

people to enter and make

a living from agriculture”

Chiji Ojukwu

The Feed Africa initiative has defined

8 areas of priority:

1. Self-sufficiency in rice. We produce

only 13 million metric tons, about 50%

of the rice that we need. Feed Africa

sets a 5-year goal for rice production

of 26 million metrics tons.

2. Increase cassava productivity from

10 metric tons/hectare to 16 million

tons/hectare, and add 52 million tons

by 2025.

3. Food security in the Sahel, with sufficient

livestock and cereal production.

4. Unlock the potential of the African

Savanna, inspired by Brazil’s transformation

of its Cerrado into a food basket

for the world by focusing on cereals,

and increasing maize production from

1.8 million to 5.4 tons/hectare.

5. Revitalize our tree crops: cocoa, coffee,

cotton, and cashew nuts, among

others.

6. Promote horticulture and produce

our own vegetables to save the

$16 billion we currently spend on

imported vegetables.

7. Grow to 1.7 million commercial farmers

producing wheat.

8. We need 5 million metric tons of fish to

become self-sufficient, as opposed to

the 2 million tons currently produced.

25

Evaluation Week 2016


Feed Africa

The AfDB Feed Africa initiative seeks to

address four main issues: poverty, malnutrition,

food imports, and food transformation.

420 MILLION

PEOPLE

LIVE BELOW THE POVERTY LINE OF

$1.25/DAY

AND THE NUMBERS ARE RISING

AFRICA DOES NOT PRODUCE

ENOUGH FOOD TO FEED ITSELF

AND IMPORTS A NET VALUE OF

$35.4 BILLION

OF FOOD EVERY YEAR

OVER 30%

OF AFRICAN

CHILDREN

SUFFER

FROM

MALNUTRITION

AFRICA PRODUCES 75%

OF THE WORLD’S COCOA

YET ADDS LESS THAN

20%

TO ITS VALUE

THE CHOCOLATE INDUSTRY IS WORTH

$100 BILLION/YEAR

BUT AFRICA’S

SHARE IS

CURRENTLY

ONLY ABOUT10%

26

eVALUation Matters / Fourth quarter 2016


Policies and the

enabling environment.

To achieve these objectives, we must

ensure that there are incentives for

young people to enter and make a living

from agriculture. We should also design

programs which take technology into the

field to help farmers. The AfDB has set up

a program to bring young people into agriculture,

and an affirmative action fund to

provide women with the resources they

need to get into agriculture.

Given that the cost to Feed Africa as

described here will be between $30-40

billion/year, we need to establish programs

to catalyze financing. The AfDB seeks to

lend $2.4 billion annually, which will reach

$24 billion annually by working with partners

and through the leverage effect.

“Local stakeholders must

be empowered to invent

the solutions to overcome

the challenges”

Fadel Ndiame

Some significant changes currently

give hope that agriculture can actually

be at the center of transformation

in Africa. First, there is highest-level of

political commitment to put agriculture

at the center of our development

and an accountability framework that

tracks progress, naming and shaming

those who do not perform well. There is

also significant donor interest and more

commitment than 10 years ago, now

that agriculture is back on the agenda.

Agriculture probably has the best prospects

of ending poverty and creating

wealth. 70% of Africans work in a sector

with significant yield gaps and income

gaps that we can fill. The private sector

is the third game changer, since agriculture

can now attract investment, create

employment and wealth.

27

Evaluation Week 2016


Feed Africa

AGRA believes that transformation must

be innovation-driven, smallholder-centered

and market-led. We must listen to

smallholder farmers and women farmers,

giving them the choice of which crops to

grow and sell. Local stakeholders must be

empowered to invent the solutions to overcome

the challenges. This requires building

systems that can respond to the needs

of smallholder farmers, creating market

opportunities, and having supportive

national policies. We need to rebuild agricultural

systems that attract young people

to agriculture and use technology to help

produce the quality and quantity of food

we need. We also need to rethink delivery

systems to incentivize farmers and adopt

policies that enable farmers to anticipate

price fluctuations and avoid seeing their

profit margins eroded.

“You cannot transform agriculture

without energy, infrastructure,

investment in human capital,

education, and the overarching

integration process”

George Mavrotas

A very substantial part of the population in

Africa works in the rural sector and a very

substantial number of these people live

below the poverty line. Investing in agricultural

productivity will improve employment

opportunities in rural regions.

Apart from comprehensive vocational

training, the agriculture education

curriculum in lower and higher education

institutes in Africa needs to be revamped

to better prepare students for jobs in agriculture

and make the sector attractive.

Gender programs should be well targeted

to ensure that women are included in all

aspects of agriculture: access to land,

inputs, markets, technology, extension

services, and finance, etc.

We also need to see a change in donor

investment trends in agriculture. The

agricultural sector policy needs to be part

of a bigger picture that considers other

economic sectors. You cannot transform

agriculture without energy, infrastructure,

investment in human capital, education,

and the over-arching integration process.

Institutions and policy mechanisms and

policies must be strengthened by aligning

national policies to the Feed Africa

agenda and mobilizing political will and

resources.

28

eVALUation Matters / Fourth quarter 2016


Knowledge Café

Who should feed Africa and how should

Africa be fed? How can evaluation help in

the design and better implementation of

interventions to feed Africa?

Participants were unanimous on the need

to raise awareness, particularly in rural

areas, on the imperatives of monitoring

for development effectiveness, yet participants

concorded that most governments

do not have adequate evaluation structures

in place. There is a need to carry out

an evaluation of Africa’s potential in food

production and conservation. If results

show that Africa has this transformation

potential many issues will be solved.

At the design and project implementation

stage, a participatory approach would

ensure that the needs of the beneficiaries

are heard. Participants voiced their

concern that most projects are designed

with the interest of the donor rather the

community at the forefront, resulting

in ambitious yet unachievable projects.

The lack of sensitization on the impact of

projects drastically reduces ownership

by beneficiaries and therefore impacts

the sustainability of the programme.

Participants agreed that the solutions to

feed Africa go beyond programmes for

agriculture. There is a need to involve

governments, the community and the

private sector in planning resilience and

food security.

Monitoring and Evaluation experts are

often lacking during the design phase.

The participants feel that there is a strong

need for quality baseline data as a basis

for measuring performance. The baseline

allows the measurement of successful

parameters that are used as indicators.

Without the inclusion of all relevant

factors at the baseline phase the evaluation

fails to capture all important aspects

of the projects including deliverables and

failures. Also, cultural, traditional and

anthropological aspects are lost when the

project puts emphasis on quantitative

data. The action points noted by participants

include carrying out evaluability

studies prior to project implementation.

Evaluation should consider sociological

aspects of the community.

29

Evaluation Week 2016


Feed Africa

Foreground, left to right: Kako Nubukpo talking to session

moderator Erik Nyindu Kibambe, News Director Voxx Africa.

Background left to right: Antonin Dossou, Batio Bassière.

30

eVALUation Matters / Fourth quarter 2016


Leadership matters

Kako Nubukpo, Economic and Digital Director, IOF;

Former Minister of Togo

“We, as African leaders, have a legitimacy issue. We are not the spokesmen

of our societies. We need to build a shared vision, otherwise our evaluation

exercises will be branded as arbitrary. How can we forge trust between

pan-African institutions and the AfDB? Between the elite and the population?

Between the members of the population themselves? We must start

by making political leaders aware that evaluation is not like an audit. More

than the outcome of an evaluation, what is important is the learning process

which accompanies it.”

Antonin Dossou, Director at the BCEAO;

Former Minister of Benin

“In my view, there are three conditions to be fulfilled for the success of the

High 5s. Firstly, we must improve the leadership capacity in our African

states, at all levels and in all areas. Leadership must be understood as the

ability of someone to be voluntarily followed and thus to effectively influence

their collaborators to achieve common goals. The expected characteristics

of the leader are vision, ability to communicate, discipline, integrity,

humility and openness.

Secondly, we must strengthen the capacity for self-reflection in our countries

and in our institutions: we must really learn to think for ourselves and

find concepts, programs and projects adapted to the specificities of Africa.

One consequence of this is the need to strengthen research and development

in Africa at all levels and in all fields.

Thirdly, we must improve the capacity to steer and implement policies. We

are indeed very good at drafting reports but not very effective in their implementation,

and this is often due to the lack of an obligation of results.”

31

Evaluation Week 2016


https://goo.gl/d3Qblw

Industrialize

Africa

32

eVALUation Matters / Fourth quarter 2016


From left to right: Per Øyvind Bastøe, Director, Evaluation Department, Norad,

Mariam Dao Gabala, CEO of MDG Consulting, and Chair of the Board of Solidaridad,

Soraya Mellali, Executive Director, AfDB, Moderator: Sumir Lal, Director of

External Affairs, World Bank, Edward Marlow, Head of Sub-Saharan Africa Client

Group, Crédit Suisse (not visible in photo), Tim Turner, Chief Risk Officer, AfDB

“Many evaluations note contributions

to job creation but it is difficult

to attribute job creation to a

single agency’s intervention”

Per Øyvind Bastøe

Across the continent, there is little variation

in constraints to Private Sector

Development (PSD). The findings from

a Synthesis Report covering 33 evaluations

of support for the private sector

reveals that the six largest obstacles to

doing business in Africa are:

1. Lack of access to finance (for larger firms)

and electricity (for smaller firms)

2. tax rates

3. informal sector competition

4. political instability

5. corruption

6. inadequate worker and managementskills

Many bilateral donors also consider lack of or poor quality infrastructure to be

a key constraint.

Evaluation Week 2016

33


Industrialize Africa

Overall, the report found that donor strategies

were aligned with national PSD strategies,

but translating policy alignment into

selective interventions is challenging. Not

surprisingly, the evaluations clearly found

that in-country presence by donors was

vital for improving consultation with the

private sector. Donors without field presence

ran a higher risk of irrelevant design,

inadequate supervision, and poor efficiency

in program delivery.

An interesting finding is the lack of clear

documentation on the relevance of PSD

programs to poverty reduction. Many

evaluations note contributions to job

creation but it is difficult to attribute job

creation to a single agency’s intervention.

Bearing in mind that most of the

poor live in rural areas, many agriculture

and agro-business priorities and donor

programs countries in these areas were

well aligned with government programs.

The synthesis report shows that there is

a need for an updated regulatory framework

to keep up with changes in the

global trade environment. While the evaluations

provided no clear answers on the

effectiveness of reforms, several of them

highlighted political will, government

commitment, and stakeholder ownership

among key success factors.

Banks were found to be more efficient in

their capacity to deliver services than

public sector institutions, which have

long approval processes and poor design

issues. The study found mixed evidence

of the effectiveness of many of the financial

instruments but one clear finding is

that financial support was more effective

when coupled with technical assistance

and capacity building. Interventions in

targeted sectors, particularly in agriculture,

were quite effective.

Sustainability is a weak area, primarily

because it is not considered in the design

stage. When sustainability was considered

during program design, the focus

was more on the financial sustainability

of the implementing agencies than

on institutional sustainability. A capacity

assessment of a donor’s intermediaries

and the government would help donors

address the sustainability issue. Also, we

must ensure that donor support packages

include capacity building for intermediaries

designed to strengthen institutional

capacity and business support institutions

to create the right conditions for success

(specifically when targeting SMEs).

The report pointed to the need for donor

interventions to emphasize additionality

and catalytic impact. We must go

beyond “gap-filling” towards an integrated

approach focused on additionality and

catalytic impact that can lead to changes

in market structure, behavior, and to mitigate

the risk of market distortions.

Rigorous monitoring and evaluation

systems focused on outcomes and integrated

throughout the life cycle of PSD

interventions are important to design,

implement and invest.

34

eVALUation Matters / Fourth quarter 2016


Massogbè Toure-Diabate Chairperson,

Women Entrepreneurship Committee,

Confederation of Large Businesses in Côte

d’Ivoire (CGECI)

“I own an enterprise that produces,

processes, and sells cashew nuts.

While this puts me in the agro-industry

sector, I would say that my enterprise is

more agricultural than industrial. Mine

is an enterprise that employs nearly

800 employees, of whom 90% are

women, and it is their energy which has

made Côte d’Ivoire the leader in the

production and export of cashews”

To paraphrase various ministerial declarations,

the lack of cooperation between

government and the private sector is slowing

the rate of Africa’s industrial transformation.

Vietnam, after only 15 years in

the industry, has almost topped us in the

almond industry. It produces, processes

and imports from Africa to transform

in-country, creating employment, wealth

and financial gain from the final export

product. We may draw inspiration from

these governments whose strategic decisions

have boosted industries.

The private sector is the motor of development

and must be heard if governments

are to design appropriate policies

for sustainable industrialization. African

governments lack the capacity to establish

industrialization without the private sector,

and should take the measures necessary to

establish partnerships between the public

and private sectors and to mobilize the

necessary resources to invest in energy

and infrastructure.

For Africa to be industrialized, businesses

must have institutional support and mechanisms

adapted to each sector. We need a

business climate that encourages processing,

fiscal exemptions for the materials

to industrialize, financial incentives for

processing, incentives to extend industrialization,

and easier access to finance.

The other brake on our industrialization is

the lack of trained labor and poor coordination

between training and jobs. We need to

examine our human resources and better

align our education systems with our labor

market needs.

We need a high-quality finished product

for a demanding client, which requires

qualified labor. We need better information,

training, and government support to

replace poverty with added value. Most of

all, Africa needs effective strategies and

concrete actions to effectively transform

our natural and human resources into

sustainable industrialization.

35

Evaluation Week 2016


Industrialize Africa

“We must start by redefining

industrialization and by

meeting our own needs”

Mariam Dao Gabala

We cannot move towards African industrialization

unless we challenge the

current models, beginning with our own

definitions of industrialization. We can’t

move towards globalization if we can’t

meet the needs of African countries and

people. Africa should not industrialize

first and foremost to export; we must

start with internal needs.

In addition, we must change how industrialization

is financed. We are too reliant

on external assistance for industrializing.

People from the outside can only bring

us what they know works in their environment,

but this may not be adapted

to what we need or want. We can take

micro-finance as an example. We too

often say that we’re too poor and without

resources, yet we know that the

poorest people save the most and know

how they want to use savings. We must

generate resources internally ourselves

and challenge our central banks, using

the example of micro-financing models.

Let’s innovate and generate our own

resources. If Africa continues with the

same thinking and financing model for

its industrialization, we risk repeating the

same mistakes. We need to find solutions

that are adapted to us.

36

eVALUation Matters / Fourth quarter 2016


“The continent now has the means

to launch a new generation of

industrialization, and countries

with a head start can lead the way”

Soraya Mellali

Africa has de-industrialized over the

last four decades, moving steadily

away from industrial processes. Africa

can catch up and industrialize quickly

by drawing on lessons from the Asian

success stories, where stable governments

have reassured investors and

where constructive policies have

opened markets and fixed favorable

trade and currency rates.

The continent now has the means to

launch a new generation of industrialization,

and countries with a head start can

lead the way. Ethiopia, Tanzania, Ghana

and Côte d’Ivoire are some examples.

We should be promoting regional value

chains helping local industries which

produce manpower-intensive products

to move up the value chain.

Changes in the global economy also

present opportunities for the continent.

As China moves away from its role as an

exporter to one as a consumer and an

investor, for example; we can take advantage

and find new ways to fill the gap and

become part of global value chains.

Despite the numerous opportunities with

China and at the regional level, free trade

and the international fragmentation of

production value chains, the continent

still has to overcome major industrialization

hurdles. Poor infrastructure reduces

productivity and the inherent costs of

poor roads, transport, water and connectivity

for example are too high. If African

industries are to become competitive,

considerable national investment in infrastructure

is needed.

37

Evaluation Week 2016


Industrialize Africa

Edward Marlow, Tim Turner

(L to R)

“The biggest impediment to industrialization

is that governments are

unpredictable in their policy responses

to a whole range of enablers – physical

infrastructure, capital markets,

and human development”

Tim Turner

Edward Marlow

Providing infrastructure is critical to

enabling industrialization. Ethiopia has

very much focused on building its industrial

enablers and has invested in government-led

infrastructure programs, which

assumes that the government builds and

industrialization follows.

Government enablers for the industrial

base is an example to follow. Commercial

banks of capital are risk-averse, balancesheet

constrained, and reliant on investors

who demand high returns. Development

Banks can assist by providing credit guarantees

or persuading governments to

guarantee project risks. Also, the early

involvement of quality lending mechanisms

such as the AfDB is the best way to

encourage the commercial investors and

pension funds into the programmes on a

long term basis.

Tim Turner

African governments should encourage

industrialization with appropriate policies.

They should set the rules, the regulatory

processes and create an enabling

environment, i.e. A level playing field

for the players. They should, however,

abstain from trying to define the players

in the game.

38

eVALUation Matters / Fourth quarter 2016


The biggest impediment to industrialization

is that governments are unpredictable

in their policy responses to

a whole range of enablers – physical

infrastructure, capital markets, and

human development. The recent shock

in commodity prices has revealed how

many governments responded too late

or did too little. Commodity industries

and the private sector are not the only

ones to have suffered as a result of their

poor response. Capital markets and

rating agencies also reacted negatively

to poor government reactions. There

has been a significant downgrading in

poor shock-response countries over the

past 5 years, shying financial markets

away from investing capital, and thereby

creating a downward spiral.

We need to focus our attention on creating

strong, predictable policy responses.

Development needs are far beyond the

lending capacity of most development

banks, which means that the business

model may need to be rethought from a

buy-to-hold model to seeking private investors

to share risk. Financial sustainability is

the precursor to all sustainability.

Knowledge Café

Participants were unanimous on the fact

that industrialization is the missing link

in the true transformation of Africa. The

main goal for development evaluation

should be to effectively address the African

paradox of experiencing poverty while

being rich in natural resources. There is

too much exportation of these natural

resources in raw forms without value addition

measures, and value addition benefits.

The industrial sector is the one that can

provide the strongest economies of scale,

sustainable growth and job creation.

Participants agreed on the need for a

human-centered approach, with focus on

capacity building, and gender inclusiveness.

They highlighted the need to change

mentalities and create an enabling environment

engaging the people, policies,

systems and stakeholders.

To achieve this goal, participants

suggested using innovation and appropriate

technologies as inputs and outputs

of the industrialization process, both

at large-scale project and small- scale

project level. They also highlighted that

economic integration and the availability

of electricity are key to sustain the industrialisation

of Africa. Evaluation should

include the question of good governance

and address the issue of competitive

advantage. Participants recommended

strengthening evaluation and monitoring

capacities within communities and

better communicating and disseminating

evaluation results.

39

Evaluation Week 2016


https://goo.gl/DemhIO

Integrate

Africa

40

eVALUation Matters / Fourth quarter 2016


“All the regional integration mechanisms are

extremely dependent on external funding yet

there is surprisingly little systematic evidence

about the effects of donor support and

development financing on regional integration”

Fredrik Söderbaum, University of Gothenburg, Sweden

Since the 1960s, regional integration

(RI) in Africa has been strongly pursued.

Changing objectives, strategies, and

institutional designs have led to a greater

diversity of African integration mechanisms

in a context of globalization, including

regional organizations, networks,

and functional regional programs as well

as bottom-up regionalism.

Greater diversity of regional cooperation

and integration mechanisms notwithstanding,

poor performance and the

lack of evidence-based results among

African regional organizations have led

to reduced external funding. There is,

therefore, an urgent need for systematic,

high-quality evidence from evaluation

and research. There is little evidence

regarding the mechanisms that perform

best or the development finance that

works best and has the best development

impact, or what institutional design has

the best development impact.

Development banks say that lack of funding

is not the main problem for regional

integration. Rather, they claim that the

problem is to identify and design projects

that are sustainable and/or bankable.

Even more urgent than economics and

funding, therefore, is the need to get

politics and governance right. Several

41

Evaluation Week 2016


Integrate Africa

decades of capacity building to State-led

regional organizations has not closed

the implementation gap. Organizational

problems, poor relations with member

states, countries that by-pass regional

organizations, and conflict among

member countries remain hurdles to

improved regional integration. The lack

of results frameworks and poor reporting

also create issues with donors and other

integration stakeholders.

“Political commitment and

national buy-in are important to

getting regional governance right.

Capacity in national ministries

needs to be strengthened for

national implementation.”

Nyamajeje Calleb Weggoro,

Executive Director, AfDB

Several agreements, initiatives and plans

to integrate Africa have made a difference

and impacted the lives of Africans –

the Lagos Plan of Action, the Abuja Treaty

of 1991, The West African Economic and

Monetary Union of 1989, the Common

Market for Eastern and Southern Africa

of 1994, the Southern African Development

Community of 1980, and the Intergovernmental

Authority on Development

of 1986, to name but a few.

We must begin by understanding the

nature of regional integration, which is

not the equivalent of development. In

fact, its impact is not always positive for

all stakeholders.

If we are to remove the barriers to

regional integration, we must work hard

to improve politics and regional governance,

which is currently fragmented

and overly focused on outputs at the

expense of outcomes and development

impact. For example, while advantages

of regional infrastructure projects such

as cross-border roads and corridors are

widely considered to have a positive

impact at regional level, there is a poor

record of implementation.

We must collectively determine which

regional integration mechanisms

perform best, and ensure that every

country is strongly committed to the

42

eVALUation Matters / Fourth quarter 2016


design of new programs. Public perception

of regional economic communities

and their roles is poor. Too much bureaucracy

has limited outreach to grassroots

participants and too many discussions

exclude the people who are actually

involved in business, research and work

across borders.

As we move forward, we must demonstrate

the positive results of regional integration

and its impact and ensure that

the process is inclusive and participatory

at local, national, and regional levels.

Clearly, we must define and demonstrate

the impact of regional integration to

stimulate stakeholders’ interest in it.

Knowledge Café

How can we design and implement

meaningful evaluations to support

effective, efficient integration in Africa?

Participants agreed that the object of

evaluation must be defined when evaluating

projects in the context of regional

integration, since the definition given

affects the choice of methods and questions.

The fact that regional integration is

driven by multiple factors makes it even

more important to clearly define the

object of the evaluation for evaluability

assessment purposes.

Participants first identified the following

objects of evaluation in the context of

regional integration: projects; programs;

regional economic communities (RECs),

and national policies geared towards

regional integration, etc. They subsequently

discussed the importance of

regional integration indicators and

agreed, unanimously, on the multi-dimensional

nature of the process (political,

economic, cultural, etc.) and the

need for myriad indicators. They noted

the lack of consensus on commonly-accepted

indicators of regional integration

making it important to define appropriate

indicators.

Evaluation teams need the skills to

accommodate the multidimensional

nature of regional projects. Some participants

therefore recommended including

new disciplines in the evaluation team

– environment, risk management – so

that the impacts of political, financial,

economic, operational, marketplace, and

environmental issues on regional integration

programs could be assessed.

Others recommended that new technologies

along with new skills to use them be

introduced to monitor data and real-time

data to improve implementation.

Timing an evaluation team’s involvement

with a project is crucial, and several

participants pointed out that evaluation

teams often come a little too late; they

recommended that they be involved from

the outset. Others argued that this could

compromise the independence of the

evaluators, and emphasized that it was

important to distinguish between ex-ante

evaluation, monitoring and evaluation,

and ex-post evaluation.

43

Evaluation Week 2016


https://goo.gl/DemhIO

Evaluation

Choices

44

eVALUation Matters / Fourth quarter 2016


“Lobbying for evaluation

should get everyone

involved in the process:

The Board, Management,

staff but also civil society”

Ruben Lamdany, Deputy Director, Independent

Evaluation Office of the International Monetary Fund

Self vs. Independent

Evaluation

Self-evaluation often affords more

ready access to information. It can

have a timelier impact on policies and

practices, as well as on staff learning.

Independent evaluation can question

more fundamental issues. It can add

credibility and weight from being

able to speak the truth to power.

A Lessons vs.

Recommendations Role

Evaluation offices are best placed to

draw conclusions and lessons from

evaluation findings. They can also provide

high level general recommendations.

Boards need specific monitorable

recommendations to track

implementation, which are better

proposed by IFI management rather

than the evaluation office.

45

Evaluation Week 2016


https://vimeo.com/194059265

Improve the

Quality of life

for the people of Africa

Evaluation Week Photo Contest – Second prize

Yanick Frutueux Cadnel Folly,

Bénin

46

A young man holding a flashlight in his hand, and

a bag of coal on his head, stands in the middle

of railways, in the commune of Allada in Benin.

What is the best energy mix to light up and power

Benin, and how can we industrialize processes to

improve the lives of the millions of young men like

the one in the photo, across Africa?

eVALUation Matters / Fourth quarter 2016


“How can we get rid of the

barriers of ethnocentrism?

Of poor governance? The

propensity to choose the

short term? Of easy profit?

In other words, how can build

a society based on values?”

Antonin Dossou, Director at the BCEAO,

Former Minister of Benin

Despite its strengths, potential and some

progress, Africa generally lags in development.

The African Union’s Agenda 2063

for the development of Africa aims to

remediate by building an “integrated, prosperous

and peaceful Africa, driven by its

own citizens and representing a dynamic

force in the international arena.”

We must do more and better if we are to

achieve the economic and social development

of Africa, for prosperity cannot be

built without effort, without perseverance,

without a collective spirit.

In this respect, the main challenge for the

success of the AfDB High-5s, beyond the

quality of program and project design, is

primarily political and sociological.

How can we get rid of the barriers of

ethnocentrism? Of poor governance?

The propensity to choose the short

term? Of easy profit? In other words,

how can build a society based on

values? I think that the AfDB can help at

all these levels.

47

Evaluation Week 2016


Improve the quality of life for the people of Africa

“More than 48% of Africa's

youth are unemployed,

underemployed or inactive.

These millions of young people

are sources of ingenuity and

engines of productivity that, if

cultivated properly, could ignite

a new age of prosperity. ”

Charles Boamah, VP of Finance at AfDB

Africa is the youngest continent in the world

with an estimated 60% of the population

between 15 and 24 years of age. More than

48% of Africa’s youth are unemployed, underemployed

or inactive. These millions of

young people are sources of ingenuity and

engines of productivity that, if cultivated

properly, could ignite a new age of prosperity.

The alternative is the prospect of a fragile

and frustrated generation. We cannot let this

happen. We cannot miss this chance.

In this context, the Bank will place particular

emphasis on education, while continuing

to support access to water, sanitation and

health services. For the education sector,

much has already been achieved – the

number of out-of-school children of primary

age has decreased by 26%. Furthermore,

the gross enrollment rate for secondary and

tertiary education has increased by over

40% and 50% respectively. However, educational

“inputs” continue to be insufficient.

Classrooms remain overcrowded against

global averages, with some schools adopting

a “shift” system to ensure everyone

gets the chance to go to school. UNESCO

notes that 7 in 10 countries in Africa face

a shortage of qualified teachers. From our

evaluations, we know that many schools

in rural communities lack access to critical

facilities, including ICT labs and science

laboratories, placing students at a serious

disadvantage. Upon graduation, the majority

of students enter into the informal economy,

characterized by instability, low wages

and low levels of social protection. Although

young people are more educated than ever

before, prospects for gainful employment

have not increased. Going forward, the

Bank will place emphasis on vocational and

educational schemes under the “Jobs for

Africa’s Youth” Initiative. Under the Strategy,

the Bank will seek to create rural jobs

for youth through the promotion of the

agriculture sector. We will develop Skills

Enhancement Zones that will link industrial

clusters with young graduates, who will be

further supported with business incubation

and finance facilities. Finally, we will partner

with governments and the private sector

to promote access to quality technical and

vocational education.

48

eVALUation Matters / Fourth quarter 2016


“One example of a quick win

to improve the quality of life of

urban citizens is a plastic bag

treatment project in Burkina Faso”

Batio Bassière, Minister of the Environment, Burkina Faso

We must focus on urban and local environmental

governance and above all on

education for sustainable development.

Strong action should be taken on landuse

plans, the promotion of sustainable

cities and the effective transfer of skills

and resources to local communities. One

example of a quick win to improve the

quality of life of urban citizens is a plastic

bag treatment project in Burkina

Faso, where communities and the

private sector are engaged in recycling

waste, resulting in job creation and a

cleaner environment.

49

Evaluation Week 2016


Improve the quality of life for the people of Africa

“Why do resources for

development initiatives in

Africa achieve only limited

or no positive transformation

in beneficiaries’ lives?”

Pindai M. Sithole, Centre for Development in Research

and Evaluation International Africa (CeDRE Africa)

Results-Based Monitoring and Evaluation:

A missing link in driving transformation

Why do resources for development initiatives

in Africa achieve only limited or no

positive transformation in beneficiaries’

lives? Why, despite all the natural resources

in Africa, are we still poor? Putting Results-

Based Monitoring and Evaluation (RBME)

at the center of any transformation agenda

can provide the missing link to help countries

achieve more from development.

What is RBME?

Results-Based Management (RBM) holds

that every development action should

lead to positive change/transformation in

people’s lives and requires clearly defined

qualitative and quantitative indicators

at output, outcome, and impact levels.

Results-Based Monitoring and Evaluation

(RBME) is premised on achieving transformation

and begins with a theory of

change (TOC) stating that human efforts

and resources should be used to positively

transform the lives of the intended people.

RBME is a concept and practice used to

ensure that ongoing monitoring or tracking

of project/program implementation is

systematic and logical. It must be properly

planned and budgeted for, use mixed methodology

and cultural and language-relevant/contextualized

data collection tools,

and reporting should focus on whether

positive change has occurred beyond the

initial problems and needs, and cover activities,

outputs, outcomes and impacts.

Unlike the traditional mindset that starts

with resources (input-based wiring), which

evidence has shown to limit the propensity

to achieve transformation, RBME

must start with a clearly stated result from

which people work backwards to mobilize

resources to achieve it.

Pitfalls in the design of development

initiatives

Symptoms and root causes are often

confused in needs assessments, a cardinal

pitfall that reduces the chance for transformation

in people’s lives. Understanding

50

eVALUation Matters / Fourth quarter 2016


the need and making it clear are primary

requirements for any development initiative

destined to transform people’s lives. I

offer an illustration of the danger of mixing

up symptoms and real needs from my

personal experience.

From February 2007 to October 2011,

I suffered from persistent headaches

and fatigue, low blood pressure and

a gradual loss of vision. Despite medical

attention from many practitioners

and opticians, the advice of family

members, friends, and colleagues, and

seven different pairs of spectacles, my

health deteriorated quickly.

In October 2011, I saw Dr. Nhando, whom I

had just met, for eye drops. He asked why

I needed them. I gave an elaborate explanation

about my health over the past five

years, to which he responded that something

was wrong with the way my health

issues were being handled. He referred

me to Dr. Dennis Sibanda, who made a

thorough examination and ordered an

MRI. There was a lump growing towards

my brain and obscuring my vision. Three

weeks later, I had an operation to remove

the lump. I no longer experience persistent

headaches, fatigue or low blood pressure.

I have regained my sight.

My narrative is a clear demonstration that

efforts directed at symptoms and not the

real need fail to achieve positive results.

Negative symptoms flag that something

is wrong but they are a means and not an

end. It also shows that quick, durable transformation

can be achieved if energy and

resources are directed to those needs.

Specificity in the intended

transfor mation

If designers of development initiatives

are not specific about the change they

intend to bring about in people’s lives, the

chances for failure in achieving meaningful

transformation become high. In

development initiatives, there can be

disastrous effects on people’s lives if

program design is not sufficiently specific

in its intentions for transformation.

Program designers sometimes lack specificity

in expressing the intended transformation

because they have limited

knowledge of the Results-Based agenda

in development, or they use top-down

approach which does not consult the

target group about their problems and

needs. Their knowledge of the indigenous

language and cultural dynamics

of the society in which the program is

to be implemented may be limited, or

they may fear accountability, expressing

transformation in ambiguous terms and

making an array of excuses about why a

result was not achieved.

Involving the beneficiaries

Everyone involved in a development initiative

must possess the required competencies

and have a heightened sense of

urgency to enable a synergistic, coordinated

response to transformation in

people’s lives. The theory echoes African

wisdom or a moral philosophy of synergy:

“It takes the whole village to raise a child”.

Stakeholders at every level of project/

program – from funder to technical

stakeholders to implementers --should

be literate and skilled in the results-based

concept and practice, and make it an

underpinning principle throughout the

project cycle. Above all, the people whose

lives should be improved by the program

must be involved.

Evaluation Week 2016

51


Improve the quality of life for the people of Africa

Using smartphones for citizen engagement in monitoring

the implementation of public policies

“African communities can help

themselves by participating

in programs which use

smartphones to collect data

to evaluate projects.”

Miltiade Tchifou Dieffi, Cameroonian

association for the development of evaluation

Using smartphones to contribute to public

welfare contributes to a people-centered

versus a technocratic centered evaluation

of public policy. The programs must first

and foremost encourage communities to

participate in the design, implementation,

monitoring and evaluation of public policies

which can improve their lives. One of

the major challenges in this context is the

training of populations both in the manipulation

of measurement indicators and in

the use of smartphones for the collection

of data.

One example is a health monitoring

scheme, the REMMOCC Project (Reducing

the Mortality and Morbidity of Cholera

in Cameroon). The objective is to identify

potential sources of cholera in order

to carry out preventive actions through

the reporting on water points in the city of

Douala and in extreme north Cameroon.

30 investigators from these locations

were trained in the identification and characterization

of water points and the use

of smartphones for reporting purposes.

More than 1500 water points were identified

and evaluated in three months

(August – October 2016).

The use of smartphones has greatly

reduced the margin of error in data collection

because the data analyzed are those

directly entered by the interviewer. The

data collected is geolocated and easily verifiable

on the field. The processing time for

survey data is also significantly reduced.

These factors combined make data collection

by smartphones an economical and

effective way of collecting data and involving

communities in the monitoring of

public policies and programs which are

intended to improve the quality of life in

the most remote regions.

52

eVALUation Matters / Fourth quarter 2016


Knowledge Café

How can evaluations help improve the

quality of life for Africans?

Participants stressed the participatory

approach in evaluating improvements

in the quality of life. They agreed on the

fact that current evaluation processes

are not communicative or participative

enough. To obtain baseline indicators, it is

important that beneficiaries be engaged

from the outset. To achieve fruitful results,

populations must be properly informed

and engaged at evaluation design and

implementation stage, to make sure that

the evaluation outcomes are also owned

by them. Participants also noted the fact

that lack of beneficiaries’ participation in

project design may lead to lack of interest

in lessons learnt from an evaluation exercise;

thereby compromising sustainability.

To achieve this participation, the process

needs to include a storyteller or key

persons within the community (traditional

chief for example). Evaluations

should have populations as their main

target and should be context and culture

sensitive. Participants highlighted the

importance of communicating the evaluation

results through a variety of media

and in local languages.

Political willingness to take forward evaluation

recommendations is critical to

achieve change. The productive use of

results depends on the implementation

of the recommendations from the evaluation

findings, yet decision makers often

fear change. Participants voiced their

experience of evaluation results and

recommendations not being properly

used. Evaluation outcomes should be

transparent and not serve the purpose of

marketing or publicity. Likewise, evaluations

and development decisions should

avoid political influence on the projects

and programs. Efforts in capacity building

and strengthening the national

statistical systems should be made.

Participants suggested the allocation of

resources not only to carry out evaluations,

but also to implement recommendations

that come from these evaluations.

53

Evaluation Week 2016


The Evaluation Week contests

From left to right: Rakesh Nangia, IDEV Evaluator General,

Jean-Luc Dawoulé Bohoussou, Senator Roger Mbassa Ndine.

54

eVALUation Matters / Fourth quarter 2016


The Evaluation Week contests

The photo contest

Photo entries were judged on creativity,

quality, originality, relevance for

the theme of Evaluation Week 2016, and

overall impact in relation to the evaluation

story. They were displayed on giant

screens in the AfDB lobby during Evaluation

Week. The winner and two runners

up received a trophy and a certificate at

the Awards Ceremony and a bursary to

attend the Evaluation Week.

The essay contest

Essays were judged on the quality of the

technical content, the relevance for the

theme of Evaluation Week 2016, and

overall impact in relation to the evaluation

story.

The winner and two runners up received

a trophy and a certificate at the Awards

Ceremony and a bursary to attend the

Evaluation Week.

Essay winners from left to right: Oluwasegun Seriki, Amali Abraham Amali,

Yao Roger Modeste Apahou, Yannick Folly (photo contest runner-up).

The essays can be found at https://goo.gl/d3Qblw

55

Evaluation Week 2016


56

eVALUation Matters / Fourth quarter 2016


Evaluation Week Photo Contest – First prize

Jean-Luc Dawoulé Bohoussou, NGO MESAD,

Abidjan.

A child is struggling to climb the first steps of a

ladder, while a schoolgirl is patiently waiting at the

top of it. A child has to work hard to succeed.

Our association, “MESAD” opens its doors for children

of all walks of life. By helping them climb social

and learning ladders, MESAD plays its part to

“Improve the quality of life for the people of Africa”.

The photo was taken in the one of MESAD’s care

centres for children in the commune of Koumassi

in Abidjan, during a homework assistance session.

57

Evaluation Week 2016


Q&A on Impact Evaluation in International Development

https://goo.gl/DemhIO

Impact Evaluation in

International

Development

58

eVALUation Matters / Fourth quarter 2016


Abebe Shimeles, Acting director of the

Research Department (EDRE), AfDB

“For subtle development relationships,

such as determining

whether there is an impact on

childhood nutrition during an

inflationary period, IE helps

measure the causal relationship.”

Quick-fire Q&A with Abebe Shimeles

1. Why do we need Impact Evaluation in a development institution such

as the AfDB?

A well-designed IE can answer questions about program design – about which parts

work rather than just whether or not a program works as a whole.

2. If evaluation is an important tool and exercise, why haven’t we been

doing it or what is our experience using it?

The high cost of IE precludes its use for all programs. An institution such as IDEV

must be selective, focusing on pilot programs which need to be scaled up, on interventions

that do not provide surface evidence of impact, and on a selection of

interventions across the Bank’s portfolio.

3. What can we do to mainstream impact evaluation in our programs?

To overcome resistance to impact evaluation, an institution must develop a culture

of evaluation whether by using internal regulations or by creating popular demand.

We need senior management commitment. If no-one is forced or mandated to do

it, few operational colleagues will agree to engage in a rigorous IE on a voluntary

basis, given the cost and time to conduct them.

Evaluation Week 2016

59


Q&A on Impact Evaluation in International Development

On tangible benefits

for governments

Where the causal link between interventions

and outcomes is straightforward,

as for example when distributing food

to refugees as part of emergency assistance

in a war-torn country, the impact

on malnutrition is obvious; this kind of

situation does not require a major impact

evaluation. For subtle development relationships,

however, such as determining

whether there is an impact on childhood

nutrition during an inflationary period,

IE helps measure the causal relationship.

Sanitation is another example. Everyone

knows that improved sanitation provides

positive outcomes, but water and sanitation

projects occasionally lead to the

opposite results. There are many examples

of water points that have been built

where people have resisted and not used

them, or that have not been well built and

are not functional.

Policy dialogue for peer learning is most

important. We can apply whatever we

learn from IE in one country to another.

For example, we know that tax evasion

among businesses is very common

across Africa, but we know neither its

magnitude nor the type of intervention

that could make companies pay their

taxes on a timely basis. We took Ethiopia

as a test case and made an analysis of

tax evasion. It revealed that more than

40% of Ethiopian companies avoid or

evade taxes, which is a huge burden for

the government. We then evaluated the

effectiveness of different ways in which

the government could contact businesses,

and tested various instruments

to see what type of pressure would make

them more responsive to tax bills. Not

so surprisingly, the threats of an audit

worked well in obtaining a response. But

we also discovered that persuasion also

works very well. If you engage taxpayers

in a trusting relationship with the government,

businesses become willing to pay

their due taxes.

Another example comes from Rwanda,

where there is health insurance for social

categories including the unemployed,

farmers, and casual workers. 90 per-cent

of the Rwandese population is covered, a

far higher percentage than in most countries.

However, there was some suspicion

as to whether the healthcare scheme

was helping national development. We

conducted a study and established a

strong causal relation between the insurance

being issued for these social categories

and health outcomes.

On the cost of IE

Technologies are opening up new opportunities

to carry out IE at a reduced cost.

Africa has considerable unexploited data

that can be used to do very good IEs.

Moreover, other institutions are collecting

very interesting data on Africa. For

example, NASA Night Light data gives us

enormous information about GDP activities

worldwide, from which we can infer

answers to many development questions.

The US government undertakes

demographic and health surveys virtually

everywhere in the world that make

it possible to combine household level

and community level information. If you

want to measure the impact of water and

sanitation program on health outcomes,

for example, you can use existing data

in a given location to establish causality.

This puts us well ahead of where we were

a decade ago with respect to data techniques,

methods, and the availability and

eagerness of policymakers to learn from

others’ experience.

60

eVALUation Matters / Fourth quarter 2016


“The high cost of IEs typically

forces project managers to make

difficult budgetary choices.”

Patrice Bosso, UNICEF, Côte d’Ivoire

Political leadership is very important for

an evaluation of the impact of programs.

So is the commitment of political leaders

to responding adequately to an evaluation.

The reality in West Africa, where my

experience lies, is that political commitment

to evaluation is quite weak. In fact,

we have undertaken many evaluations

to which there has been no political

response at all.

In addition to the problem of limited

political leadership, financing for monitoring

and evaluation in general is typically

inadequate. The high cost of IEs

typically forces project managers to

make difficult budgetary choices. Moreover,

there is often weak national capacity

and the absence of a critical mass of

evaluation expertise, which can compel

countries to recruit international expertise.

Taken together, these reasons help

explain why IEs are so little used in international

development.

There is also the practical problem that

national development plans, programs,

and projects often lack a theory of

change or a results framework. Without

these, what can we evaluate?

To encourage or instill IE at country

level, we must produce evidence of its

contribution to inform policy dialogue.

We must also help countries to legislate

in favor of IEs so that governments are

obliged to conduct them. We should also

invest in capacity building to reach a

critical mass of national evaluators and

in proven information and monitoring

systems, without which evaluation costs

rise further.

UNICEF is working to ensure that country

M&E policies include measures to

strengthen national monitoring and

data production systems and to incorporate

IE in national plans and cooperation

programs.

61

Evaluation Week 2016


Q&A on Impact Evaluation in International Development

“Development is different in a

poor country where many things

need to happen simultaneously

before positive impact occurs.”

Zenda Ofir, President ICED

Working towards impact is a mindset that

is essential for development. However, evaluating

impact is useless unless you understand

how, why, for whom, and under what

circumstances change happens. In other

words, we must understand how impacts

come about, how unintended impacts

happen and may neutralize the good that

has been done. Moreover, we must understand

whether impacts will have knock-on

effects for sustained development. We

must also account for development trajectories,

but we cannot predict when change

will happen, which makes the timing of

impacts extremely important.

If you think about development in a rich

country, evaluation is not a very important

topic, since change occurs within

a stable advanced economic and social

environment. Development is different in

a poor country where many things need

to happen simultaneously before positive

impact occurs. IE is therefore a way of

understanding how development happens

in a complex environment. Evaluators work

in a context where the issue is less about

62

a project or a program than about the

connections between them.

Development and evaluation require recognizing

the interconnectedness of things

rather than seeking a predicted impact at a

particular time. We must understand where

we want to go but we also need flexibility

along the way to account for results. There

is an art to doing IE in a larger portfolio to

understand how to adjust quickly while

implementing.

There have been many approaches to

doing IE and we are still exploring methodologies.

But in addition to experimental,

quasi-experimental, and non-experimental

designs, we must look at the range of

methodologies that are useful in context.

Participatory Impact Evaluations, Realist

Evaluations and Rapid Impact Evaluations

are some examples. What are the

associated constraints, and which methods

promise the best results in specific

contexts, for specific purposes, and at

appropriate times?

eVALUation Matters / Fourth quarter 2016


No one has found a magical recipe for good

impact evaluations. No single methodology

or design meets all the requirements for

rigor. IEs that use experimental designs, and

especially randomized and control groups,

are useful in context, but are constrained by

the limited number of variables and, therefore,

interactions. Complex packages, small

sample sizes, expensive in-depth gender

work, spillover effects and contamination all

pose serious problems. Policymakers do not

ask a single question about a single program.

Rather, they want to understand the dynamics

of developing an entire sector.

“A participatory approach

involves all stakeholders in

the process and gives participants

a sense of ownership. It

also reduces IE costs, since

beneficiaries collect, report

data and find solutions.”

Maria Sophia Aguirre, Catholic University of America.

To rigorously systematize impact evaluation,

we must combine stratification and

randomization consistently over time, in

randomized control trials (RCT). Institutions,

human and social capital, and policy

must all work together. My methodology

suggests that we focus on actions that affect

us, on their immediate effects, and on how

they change lives. I typically focus on four

factors: quality of life, social responsibility,

civic responsibility, and social skills.

Effective evaluations use RCTs to expand

their reach to capture the interpersonal

dynamics that affect outcomes. For example,

we know that companionship and

social networks are effective channels

for outcomes. We also know that social

network theory has highlighted the need to

understand relationships and friendships in

order to understand economic outcomes.

All of this has shifted the focus from individual

maximization to a relational context

where we see how others’ decisions and

actions affect our own.

We have developed a participatory index to

measure changes in behavior and attitude

from passive to active involvement, using

participatory rather than financial incentives.

"A participatory approach involves all

stakeholders in the process and gives participants

a sense of ownership. It also reduces IE

costs, since beneficiaries collect, report data

and find solutions." Participants take responsibility

for outcomes. After the money ends,

the participants continue to practice the

solutions. This way of working contributes

to sustainable development.

63

Evaluation Week 2016


https://goo.gl/d3Qblw

Climbing the

High 5 curve with

independent evaluation

64

eVALUation Matters / Fourth quarter 2016


From left to right: Moderator Simon Mizrahi, Director, Results & Quality Assurance

Department, AfDB (missing from photo), Zenda Ofir, President, ICED, Rakesh Nangia,

Evaluator General, AfDB, Indran Naidoo, Director Evaluation Office, UNDP, Oscar

Garcia, Director Independent Office of Evaluation, IFAD, Keith Leonard, Development

Evaluation Expert, EBRD, Saphira Patel, Evaluation Department DBSA.

“Do we learn more from success or from failure?

From success, we can deduce that we should do

more of the same where failure can lead to innovation.

The private sector handles this quite well,

but in a public-sector framework, risking public

money on innovation raises issues.”

Keith Leonard

Why do we need

independent evaluation?

The moderator opened by stating that

until recently, most evaluation units

across the world were not independent

and their activities were restricted by

administrative and human constraints.

This occasionally led to conflict between

evaluation units and institutional

management and often adversely

affected work and findings.

Panellists agreed that the independence

of evaluation units has many advantages:

it helps to focus on the mandate rather

than on management and provides a

different perspective on a project or

program. This requires an independent

budget and an independent team and

striking the right balance of involvement

to avoid isolating evaluation functions.

65

Evaluation Week 2016


Climbing the High 5 curve with independent evaluation

Evaluations by units that are disconnected

from the rest of the organization

will not be endorsed and their results will

be ignored. The organizations leading

the evaluation must own the concept,

and support and be part of it. In addition,

having external peer reviews of evaluation

reports helps the credibility of evaluation

results. To avoid any conflict of

interest, self-evaluation is a necessary

first level of feedback and an external

view that wasn’t part of the implementation

or design tells you whether or not a

project works.

An ex-post evaluation ensures that we

learn from past mistakes. Both self and

ex-post evaluations play an important role

in identifying lessons.

How should IDEV

proceed in evaluating

AfDB’s High 5s?

Panellists agreed that IDEV will need

to develop appropriate approaches for

the evaluation of the High 5s, which are

complex and will be implemented in a

dynamic environment involving multiple

donors and stakeholders. Just as the Bank

works with others to finance and implement

programs to achieve the High 5s,

IDEV should engage with other donors and

conduct a joint evaluation.

The evaluation of outcomes often points

to serious gaps among programme activities

that limit the possibility of structuring

a logic model or theory of change. Sustainability

and impact must be considered at

the evaluation design stage. Evaluation

does not allow for either predictability or

rigidity but rather requires adjustments

and adaptability. It is a challenge to remain

responsive to evaluation needs, ie. adjusting

the Work Program, and to adapt to

available financial resources, for example.

How can we foster a

culture of evaluation?

Creating and embedding a culture of

evaluation is more than having an evaluation

unit. It involves buy-in at all levels

and resources. Panellists agreed that

we need to eliminate the perception

that evaluation is a threat. You also need

enough capacity, without which you do

half the job with an evaluation that picks

up some outcomes and outputs and gives

a lop-sided or biased result. There must

be people who convey the findings of an

evaluation in the right way and with good

will, telling truth to power.

Sometimes, evaluations provide

difficult messages or dare us to

think differently about our work.

Being open to these messages is

challenging for any professional.

Nobody likes to have their hard

work put under the microscope!

But evaluation is not just another

accountability tool – fundamentally,

evaluation is about learning from

experience and improving as an

organization. Achieving better

results for Africa is a challenge we at

the Bank all can and should rise to.

Charles Boamah, VP Finance, AfDB during his

opening address to Evaluation Week participants

on Tuesday 08th November 2016.

66

eVALUation Matters / Fourth quarter 2016


We have a responsibility to build evaluation systems and a

culture in the Bank’s regional member countries because

the Bank’s lending program is a very small part of the any

country’s public investment program. Our ambition should

therefore be to improve the outcome of the entire development

investment, and not just our initiatives as a Bank.

Rakesh Nangia

Panellists agreed that we should build

the demand side, making governments

accountable and thus inclined to invest

in evaluation. Evaluation is conducive to

improving people’s lives through transparency,

accountability and legitimacy.

Enlightened parliamentarians should use

evaluation results to make informed decisions

for the benefit of the nation.

What is a theory of change and why it is useful?

“A theory of change is a way of moving

away from the tyranny of linearity. It is

basically the logic of change, establishing

why we think change will happen

if we do an intervention, and can be

done when designing an intervention or

retrospectively.”

Keith Leonard

A good results framework focused on

outcomes is commendable, but is the

underlying theory of change that gives

rise to the outcomes sound? Are the

assumptions the right ones? Have the

risks been correctly identified? Prioritization,

or distinguishing pre-conditions

from conditions is an important aspect

in a theory of change. It all boils down

to a good understanding of the problem

you are trying to solve. Understanding

that underpins the theory of change and

if you cannot describe it, you don’t have

a baseline.

67

Evaluation Week 2016


Climbing the High 5 curve with independent evaluation

Three Pieces of Advice for Evaluators

Rakesh Nangia

1. Stay humble: we all make mistakes.

2. Focus on quality as much as possible.

3. Put yourself in the other person’s shoes. Operational task managers

plan their interventions ahead and evaluators come in hindsight.

Indran Naidoo

1. Recognize ourselves as professionals.

2. With power comes responsibility.

3. We achieve more by challenging than by validating.

Oscar Garcia

1. Evaluation is changing. An evaluator must be constantly learning and

keep up to date.

2. Join forces. DFIs should work together to co-produce evaluations

3. The way we give feedback on results is important

Saphira Patel

1. There is always another side to every story.

2. Be flexible and willing to change.

3. Aim for an activist reformist agenda and make sure that each

evaluation contributes to change, adding value.

Zenda Ofir

1. Relevance criteria and questions are extremely important and should

be more critical.

2. Do not be simplistic or use baseline indicators just because the data is

easily available.

3. Activities are inter-connected. Look carefully at Randomized Control

Trials and a contextual analysis that take culture and the historical

background into account.

68

eVALUation Matters / Fourth quarter 2016


There are missing links in measuring outcomes at the

UNDP that raise serious concerns. When interventions

need to be sequenced in a certain way, there is no evaluation

about how they are linked together.

Indran Naidoo

Lessons from UNDP

Independent Evaluations

A more integrated approach to development

and building on sectoral synergies

is critical for enhancing outcomes

and achieving more sustainable results.

Complexities of integrated programming

at the country level need to be

addressed and will be a critical factor in

moving forward in achieving SDGs.

UNDP’s current strategic plan takes an

integrated approach to development

support. This is due in part to the recommendations

of several IEO evaluations

to move away from a siloed approach.

The integrated approach aligns with the

SDG approach and is a clear departure

from the compartmentalized focus of

the MDGs on outcomes – poverty reduction

and the environment – that paid

little attention to the underlying factors

of poverty reduction or environmental

management, which often led to unsustainable

consequences.

An inclusive approach to

reducing poverty

Poverty reduction outcomes are

enhanced by an inclusive approach.

Growth and inclusiveness must be

pursued simultaneously. UNDP has been

effective in embedding the poverty reduction

agenda from the multidimensional

perspective of human development in

national forums for debate and discussion

about socio-economic development.

But ethnic, geographic, political and

cultural factors in some countries have

made it difficult to understand the broad,

multidimensional concept of human

poverty. UNDP still attempts to find ways

to increase attention to the centrality of

poverty reduction in its many dimensions,

and uses focused advocacy with its central

government partners and/or increasing

the space for civil society or decentralized

government structures to give voice

to specific needs and concerns, which

frequently incorporate social issues.

Interrelated policies and legislative frameworks

are important for poverty reduction

and environmental management.

National development plans and policies

related to climate change, environment,

water management, coastal area/resource

management, energy management, land

use and urban planning are critical to a

multidimensional approach to building

resilience. There are cost implications for

considering poverty reduction from this

wider perspective of reducing vulnerability,

yet most countries lack a cost-benefit

69

Evaluation Week 2016


Climbing the High 5 curve with independent evaluation

analysis of the advantage of vulnerability

reduction in addressing poverty that

could inform government decisions and

resource allocation.

Dependence on external funding, especially

for the environment and poverty

reduction, has reinforced the institutional

focus on working within thematic areas

and diverts attention from UNDP’s priorities

to those of donors. Few external funding

sources provide incentives to address

poverty-environment linkages, although

environment program funders tend to be

receptive to the initiatives of programme

units that proactively pursue integration.

Governance is poorly integrated

Generally, governance is integrated

into UNDP development projects and

programmes. About 50 percent of the

countries have developed capacity

development plans and 20 percent have

advanced implementation. This is an indication

of improved governance, oversight,

accountability, transparency and the

integrity of health systems pertaining to

Global Fund areas.

Governance performance data at country

level is limited, and UNDP relies on international

data and perception surveys.

UNECA’s 2016 African Governance Report

covers the issue of measuring corruption

and raised similar issues. It argues for

moving beyond current corruption indicators

to assessing corruption in a broader

African governance context and replacing

perception-based measures with

fact-based measures that reflect the international

dimensions of corruption. This

requires using a mix of indicators.

Leveraging development data

Few national statistical systems can

effectively meet the demands of development

data. Lack of adequate data and

of learning about what works and why

compromise the contribution of national

development programmes.

Several international agencies support data

generation and national statistical systems,

but coordination remains weak. Reporting

arrangements are often multiple and lack a

strong national statistical system that coordinates

development statistics.

Another challenge of data and analysis

gaps is that available data is not analyzed

for monitoring programme progress or

quality assurance. The MDGs improved

data and its use in reporting but this was

not used to monitor the MDGs.

Lack of capacity and resources is the most

important reason for not using available

data. National statistical systems are not

yet making the best use of available technology

to better enable development data

and analysis.

70

eVALUation Matters / Fourth quarter 2016


Evaluation Week 2016

71


Program

AfDB Development

Evaluation Week 2016

Driving Africa’s Transformation

7–10 Nov 2016 – Program

09:00–12:00 Evaluation Capacity Development Workshop

10:00–10:30 Coffee/tea break

12:00–14:00 Lunch | Lunchtime talk

Rubén Lamdany, Deputy Director, Independent Evaluation Office, IMF

14:00–16:00 Knowledge Café

18:00 Welcome cocktail (Pullman Hotel)

Monday 7 November 2016

Tuesday 8 November 2016

Delivering On The High-5S: What Will It Take? | Light Up And Power Africa

09:00–09:10 Welcome remarks

Rakesh Nangia, Evaluator General, AfDB

09:15 – 09:30 Akwaba ceremony

09:30 – 09:45 Opening Remarks: Evaluation. A Critical Link in the AfDB’s Learning Ecosystem

Bright Okogu, Executive Director, AfDB

09:45 – 10:15 Launch of AfDB Evaluation Week: Transforming Africa through the High 5s

Charles Boamah, Vice President Finance, AfDB (Representing AfDB President Adesina)

10:15 –10:30 Coffee/tea break

72

eVALUation Matters / Fourth quarter 2016


10:30–12:30 High-level Discussion: Opportunities and Challenges for Delivering on the High 5’s

Moderator: Erik Nyindu Kibambe, News Director, VOX Africa

Introductory remarks

H.E. Nialé Kaba, Minister of Planning & Development, Côte d’Ivoire

Panelists:

H.E. Batio Bassière, Minister of Environment, Burkina Faso

Kako Nubukpo, Economic and Digital Director, International Organization of Francophonie

(Former Minister for Long Term Strategy and Public Policy Evaluation, Togo)

Antonin Dossou, Director at Central Bank of West African States (BCEAO) (Former

Minister of the Evaluation of Public Policies and Denationalization Programs, Benin)

Koffi Yao, Director of Cabinet, Ministry of Planning & Development, Côte d’Ivoire

12:30–14:30 Lunch | Lunchtime talk

Maria Aguirre, Catholic University of America, USA

14:30–17:00 Panel discussion: Lighting up and Powering Africa: On the road to Renewable Energies

Moderator: Ronald Meyer, Executive Director, AfDB

Panelists:

Marc Albérola, CEO, Eranove Group / Director of Operations, Eranove Cote d’Ivoire

Elias Ayuk, Director, UNU-INRA, Accra

Amadou Hott, Vice President, Power, Energy, Climate and Green Growth, AfDB

Aka Hyacinthe Kouassi, Advisor to Executive Director, AfDB

Vanessa Ushie, Pan-Africa Senior Policy Advisor on Extractive Industries, Oxfam

15:30–15:45 Coffee/tea break

Wednesday 9 November, 2016

Delivering On The High 5S: Feeding Africa | Industrializing Africa

09:00–11:30 Feeding Africa

Moderator: Oscar Garcia, Director, Independent Office of Evaluation, IFAD

09:00–09:30 Keynote: What will it take to feed Africa – progress, challenges, & opportunities

Chris Toe, Senior Advisor on Country Strategic Plans, Policy and Programme Division, WFP

Evaluation Week 2016

73


Program

09:30–11:30 Panel discussion: Strengthening Agriculture and Food Security

Panelists:

Daniel Alberts, Senior Manager, Agriculture and Nutrition, GAIN

Anne-Sophie Le Dain, Nutrition Manager, UNICEF

George Mavrotas, Senior Research Fellow & Program Lead, Nigeria Strategy Support Program, IFPRI

Fadel Ndiame, Regional Head, AGRA West Africa

Chiji Ojukwu, Director, Agriculture and Agro-Industry Department, AfDB

10:30–10:45 Coffee/tea break

11:30–12:30 Presentation: Results Based Monitoring & Evaluation – A Paradigm

Shift toward Driving Africa’s Transformation

Pindai Sithole, Director, Social Research & Evaluation, Centre for Development

in Research and Evaluation International Africa (CeDRE Africa)

12:30–14:30 Lunch | Lunchtime talk

Indran Naidoo, Director, UNDP Evaluation Office

14:30–17:00 Industrializing Africa

Moderator: Sumir Lal, Director, External Affairs, World Bank

Introductory Remarks:

Massogbè Toure-Diabate, Chairperson, Women Entrepreneurship Committee,

Confederation of Large Businesses in Côte d’Ivoire (CGECI)

Presentation: Findings from Synthesis Report on Private Sector Evaluation

Per Øyvind Bastøe, Director, Evaluation Department, Norad

Panel Discussion: The Way Forward

Panelists:

Per Øyvind Bastøe, Director, Evaluation Department, Norad

Mariam Dao Gabala, CEO of MDG Consulting and Chair of the Board of Solidaridad

Edward Marlow, Head of Sub Saharan Africa Client Group, Credit Suisse

Soraya Mellali, Executive Director, AfDB

Tim Turner, Chief Risk Officer, AfDB

16:00–16:15 Coffee/tea break

74

eVALUation Matters / Fourth quarter 2016


Delivering On The High 5S: Improving The Quality Of Life For Africans

| Integrating Africa | Evaluating The High 5’S | Awards

09:00–10:00 Improving The Quality Of Life For Africans

Thursday 10 November, 2016

Face to Face: Frannie Léautier, Senior Vice President, and Rakesh Nangia, Evaluator General, AfDB

10:00 – 12:00 Integrating Africa

Moderator: Sunita Pitamber, Director, Human Development Department, AfDB

Presentation: Integrating Africa: What Works and Why?

Fredrik Söderbaum, University of Gothenburg, Sweden

10:30–10:45 Coffee/tea break

10:45–12:00 Discussant/Q&A

Nyamajeje Calleb Weggoro, Executive Director, AfDB

12:00–14:00 Lunch | Lunchtime talk

Presentation: Using Smartphones for Data Collection – Citizen

Engagement in monitoring the implementation of public policies

Miltiade Tchifou Dieffi, Cameroon Development Evaluation Association (CaDEA)

14:00–16:00 Roundtable Discussion: Climbing the ‘High 5’ Learning Curve with Independent Evaluation

Moderator: Simon Mizrahi, Director, Results and Quality Assurance Department, AfDB

Panelists:

Oscar Garcia, Director, Independent Office of Evaluation, IFAD

Keith Leonard, Development Evaluation Expert, ERBD

Indran Naidoo, Director, UNDP Evaluation Office

Rakesh Nangia, Evaluator General, AfDB

Zenda Ofir, President, International Centre for Evaluation and Development, Nairobi and

Honorary Professor, School of Public Leadership, Stellenbosch University, South Africa

Saphira Patel, Manager, Operations Evaluation Unit, DBSA

15:00–15:15 Coffee/tea break

16:00–16:45 Awards: Evaluation Essay Contest | Evaluation Photo Contest

16:45–17:00 Closing Remarks

17:00 Cocktail

Evaluation Week 2016

Rakesh Nangia, Evaluator General, AfDB

75


Independent Development Evaluation (IDEV)

African Development Bank

Phone: +225 20 26 20 41

Fax: + 225 20 21 31 00

Email address: idevhelpdesk@afdb.org

Website: idev.afdb.org

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!