Ken Chomitz - SDC Climate Change and Environment Network

Highlights of the Chat on Monitoring &
Evaluation of Adaptation Approaches with
Ken Chomitz
Senior Advisor in the Independent
Evaluation Group (IEG) of the World Bank
October 11, 2010 3.30 pm
HQ Ausserholligen
"We should think of M&E as not just an add on to projects but in many cases the actual
objective of the project because it can show that what was done works and is transferable
to someplace else and can be scaled up a hundredfold".
VIDEO-Link
Overview
Evaluating Climate Adaptation Efforts: Notes
Towards a Framework... page 1
Typology of Adaptation Needs, Responses and
Evaluation Approaches... page 2
Weaving the Threads Together... page 2
First Reactions to Presentation... page 3
Adaptation and the Challenge of its Monitoring and Evaluation in the World Bank
Programme
Evaluating Climate Adaption Efforts:
Notes towards a Framework
The evaluation focusing on adaptation to climate
change is the third in a series of evaluations that seek
to gain lessons from the World Bank Group's
experience in climate change. Previous evaluations
were:
Phase I: An Evaluation of World Bank Win-Win
Energy Policy Reforms (2009)
Phase II: Mitigation or the promotion of
technologies for renewable energy and energy
efficiency in the World Bank Group (2010)
With the increasing interest in climate adaptation and
the growing financial support for adaptation efforts,
monitoring and evaluation is essential in order to
ensure that adaptation needs are met in the most
effective and efficient manner.
Learning From Evaluations
One important objective of evaluations is drawing
lessons from the past and learning from our own
experience and that of others. But what might a
development agency want to learn from the
evaluation of its climate adaptation efforts
First, whether efforts aimed specifically at promoting
adaptation actually bring the intended results in an
efficient way. This is done by applying the traditional
evaluation criteria. A second goal is to learn also from
"adaptation analogues" which are efforts that may
promote adaptation even though they are not labelled
as such. Doing impact assessments is one way to
identify specific consequences of projects, for
example how risk was reduced or what the economic
returns could be. Finally, lessons from evaluations
could inform strategic choices and give indications as
to:
How to best increase efforts in mainstreaming
adaptation
How to set priorities
Whether to work within sectors or develop crosssector
approaches to adaptation
Specific Challenges in Evaluating Adaptation
Evaluators of climate adaptation projects face a
number of specific challenges related to the nature of
climate adaptation:
How to distinguish adaptation efforts from
traditional development assistance activities like
for example water management initiatives
Adaptation efforts are young and not ripe for
evaluation with traditional evaluation cycles
Climate adaptation involves to a large extent
preparations for conditions in the future or for
disastrous events (one in a 100 year event) which
- it is hoped - may not occur for decades to come,
Although it is easier to only consider activities
within sectors (e.g. making agriculture more
resilient) evaluators should also consider cross
sector adaptation issues for the results to be
meaningful (e.g. encouraging shifts out of the
agriculture sector)
Lack of metrics for vulnerability and resilience

Typology of Adaptation Needs, Responses and
Evaluation Approaches
As projects and programmes are conceived to
address different adaptation needs, they have
different design characteristics. According to Ken
Chomitz this implies that the M&E efforts to assess
the projects must also vary. Three kinds of adaptation
needs, responses and evaluation approaches were
presented.
Current Chronic Conditions
The lack of adaptation to current conditions falls
under this category. Unsustainable groundwater
extraction or chronic floods and droughts are two
examples of a current chronic condition.
Some adaptation responses could be water pricing,
creating or strengthening water management
institutions or the construction of urban drainage
infrastructure. Such activities already have a
substantial track record of experience and therefore
evaluators can look at outcomes ex post.
Unreckonable (incalculable) Risks
As the name implies unreckonable risks are difficult to
grasp and plan for in a meaningful way. Scientific
models produce different future scenarios which
should somehow be reflected in adaptation
approaches. But how can we plan climate sensitive,
long-lived infrastructure when according to some
projections rainfall will increase whereas others
predict a decrease For example, when designing a
coastal defence against storm surges, what we
consider a one in a thousand year event might
already be a one in a hundred year event.
Two adaptation responses are to either shift away
from climate sensitive locations and sectors or to
design (costly) robust infrastructure.
For unreckonable risks, ex post evaluations are
almost impossible, as in many cases the designers of
the adaptation responses will not be around to assess
them anymore. Consequently, ex ante evaluations
should be conducted. Indeed, evaluators can
determine whether the basis for climate scenarios
and their use in the design of projects and
programmes was sound. This type of evaluation does
not require evaluators to wait for project closure and
even allows for corrections (and applying lessons
learnt) while the adaptation activities are still being
implemented.
Inexorable (unstoppable) Calamities
The third category of risk adaptation efforts to
address are inexorable calamities, such as the loss of
glacial water supplies or the inundation of low-lying
islands and deltas.
Adaptation efforts targeted at unstoppable calamities
require long lead times. For example, the construction
of reservoirs to replace natural glacial water storage
systems or the relocation of inhabitants of threatened
lowlands is complicated, highly political and
expensive. The crucial question there is when should
we start preparations for such events
As with unreckonable risks, the most appropriate type
of evaluation for inexorable calamities seems to be an
ex ante evaluation. Of course ex ante evaluations
become more meaningful if they are updated
periodically.
Adaptation Actions and Evaluation Criteria
Efforts to address the adaptation needs above can be
categorised into four main areas of action:
1) Information & Tools, e.g. GIS
2) Capacity Building, e.g. training national staff
3) Monitoring, diagnosis and planning of
activities
4) Financing and implementing plans
For the first three adaptation actions, ex post
evaluations are more appropriate. The reliability of
information & tools, their availability and their
adequacy for reaching the intended purpose as well
as their transparency can be examined. Evaluators
can request feedback from users and examine the
documentation of such adaptation actions. Similar
aspects can be analysed concerning capacity building
(use of tools). Additional matters such as the
adequacy of funding and staffing for institutional
development can be evaluated. An important criteria
with regard to capacity building is the strength of
cross-sectoral and cross agency cooperation.
Monitoring, diagnosis and planning of activities can
be evaluated with regard to their adequacy, the
degree of integration of the climatic concern into
planning, the sequence of strategic priority setting.
Financing and implementing plans can be evaluated
ex ante or ex post depending on the questions, e.g.
taking into consideration whether the plans are being
implemented or not, or the examination of impacts
versus costs.
These categories of adaptation actions are
implemented at the global, national and project levels
by development agencies or within the multilateral
development banks.
Geography as Organising Principle
"We should think of M&E as not just an add on to
projects but in many cases the actual objective of the
project because it can show that what was done
works and is transferable to someplace else and can
be scaled up a hundredfold".(Ken Chomitz)
Regarding the transferability of adaptation
approaches to climate change, the geographic
conditions of a region are paramount. Biomes and
geographic regions have distinctive bundles of
adaptation needs and solutions. Consequently,
geography can serve as an organising principle

Quantifying Impact on Resilience
There are different ways to increase resilience to
climate change. One approach currently being
pursued is an insurance system for farmers and those
in other sectors who are very vulnerable to weather
conditions. As the graph below illustrates, farm
income can be stabilised in periods of extremes such
as droughts and floods with such an insurance
system. The thick black line shows the baseline
conditions whereas the grey dashed line shows the
adaptive response with such a system - in other
words increased resilience. In times of drought or
flood the farmers receive support whereas in normal
conditions they pay higher contributions (the distance
between the lines illustrates this "buffer effect").
Farm Income
Drought Rainfall Flood
Probability of Rainfall
In general, feasible measures of increasing resilience
towards vulnerability at the national level depend to a
large extent on the existence and operation of water
resource management institutions as well as the
specific contexts in which people and assets are
exposed to extreme weather events.
Tracking Adaptation Expenditures
Current tracking efforts attempt to attribute the
adaptation share of funding for each development
project. The motivation behind this is to assess the
quantity of additional funds available as well as the
investments in adaptation efforts. However, this way
of tracking adaptation expenditures raises several
issues of concern. Do only intentional adaptation
efforts count How can the adaptation share be
allocated when adaptation is integral to the project
design and goals The emphasis on outputs rather
than outcomes in adaptation methods is not
conducive to efficiency either. Therefore, one of the
key remaining challenges is coming up with
reasonable measures for implementing and tracking
adaptation expenditures in a meaningful way.
First Reactions to the Presentation
SDC is in the process of producing a handbook to
provide guidance in integrating climate change
mitigation/adaptation and disaster risk reduction
into development cooperation. M&E issues such
as the ones discussed above are relevant here
too.
One key challenge that remains to be addressed
is whether it makes sense – and if so how – to
differentiate between development cooperation
and adaptation efforts, i.e. defining indicators for
adaptation that are not development indicators.
Also, how can we define risks if we have no clear
ideas of direct and indirect impacts of climate
change
Some suggested that maybe the issue was not as
complicated as it seems: we can define baselines
and risks and then consider all efforts that
address those risks as adaptation.
However, for political reasons it makes sense to
treat adaptation as a separate issue, as this
allows for earmarked funds (as is the case with
other issues that are mainstreamed)
A further challenge is designing adaptation
projects and programmes that do not address
adaptation in an isolated manner - social,
political, cultural, economic and other factors
should be considered
Climate Change and the World Bank Group
Phase II: The Challenge of Low Carbon
Development
The Evaluation will soon be published by the IEG
and distributed within SDC.
What works for climate & development
Energy efficiency (EE) & lighting, transmission
and distribution (T&D) loss reduction
Forest protection (where local people received
user rights)
Long tenor loans for renewable energy
Technology transfer via piloting¨
What didn’t work so far:
Carbon finance to induce more investment
into "renewables"
Loan guarantees as market transforming
instruments for EE
In conclusion, there is a need for:
More immediate feedback & data on energy
produced, forest conserved, people served and
carbon returns (kg of avoided carbon per $ of
investment costs).
Next CC Briefing: November 25, 2010
Climate Change Operations and Networking in the
UNDP (Yannick Glemarec)
Contact: ueli.mauderli@deza.admin.ch