Measuring Achievements of Private Sector Development in ... - DCED
Measuring Achievements of Private Sector Development in ... - DCED
Measuring Achievements of Private Sector Development in ... - DCED
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong><br />
<strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Practical Guidel<strong>in</strong>es<br />
for Implement<strong>in</strong>g the <strong>DCED</strong> Standard<br />
Version 2, July 2013<br />
By Ben Fowler and Adam Kessler<br />
MarketShare Associates
These guidel<strong>in</strong>es have been produced as a companion to the Donor Committee for Enterprise <strong>Development</strong><br />
(<strong>DCED</strong>) Standard for Results Measurement (hereafter referred to as the <strong>DCED</strong> Standard) and associated<br />
guidance documents.<br />
The guidel<strong>in</strong>es are based on extensive <strong>in</strong>terviews with experts and field practitioners, desk research, and<br />
two cases studies based on country visits which analysed <strong>in</strong> depth programmes’ results measurement<br />
experiences <strong>in</strong> conflict-affected environments: The DFID-funded Susta<strong>in</strong>able Employment and Economic<br />
<strong>Development</strong> (SEED) programme <strong>in</strong> Somalia and the BMZ-funded Youth Employment programme<br />
implemented by GIZ <strong>in</strong> Sierra Leone.<br />
The guidel<strong>in</strong>es have been commissioned by the <strong>DCED</strong> Work<strong>in</strong>g Group on <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> (PSD) <strong>in</strong><br />
Conflict-Affected Environments (CAEs). It was produced by MarketShare Associates (MSA) and was written by<br />
Ben Fowler and Adam Kessler. Feedback is welcome and should be sent to ben@marketshareassociates.com<br />
and adam@kessler.co.uk.<br />
For more <strong>in</strong>formation on the <strong>DCED</strong> Standard for Results Measurement, please visit the <strong>DCED</strong> website at<br />
www.enterprise-development.org/page/measur<strong>in</strong>g-and-report<strong>in</strong>g-results or contact the <strong>DCED</strong> at<br />
Coord<strong>in</strong>ator@Enterprise-<strong>Development</strong>.org.<br />
To view the <strong>DCED</strong> Knowledge Page on PSD <strong>in</strong> CAEs, <strong>in</strong>clud<strong>in</strong>g an onl<strong>in</strong>e library with more than 450<br />
documents, and guidance by various agencies, please go to www.enterprise-development.org/page/cae.<br />
For more <strong>in</strong>formation about the activities <strong>of</strong> the <strong>DCED</strong> Work<strong>in</strong>g Group on PSD <strong>in</strong> CAEs, please refer to<br />
http://www.enterprise-development.org/page/caesg.<br />
Front page photo credits (from left to right): morguefile.com; Rebuild<strong>in</strong>g <strong>of</strong> a road and bridge <strong>in</strong> Tajikistan, by<br />
Stefan Erber, GIZ; Vocational tra<strong>in</strong><strong>in</strong>g <strong>of</strong> former child soldiers who benefitted from ILO’s International<br />
Programme on the Elim<strong>in</strong>ation <strong>of</strong> Child Labour (IPEC) <strong>in</strong> DRC, © International Labour Organization/ Marcel<br />
Crozet; UN cars and cattle dealer <strong>in</strong> Afghanistan, by Jochem Theis, GIZ; Survey for ILO’s JobNet Project <strong>in</strong> Sri<br />
Lanka, © International Labour Organization/ Marcel Crozet.
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Table <strong>of</strong> Contents<br />
I. Introduction .......................................................................................................................................... 3<br />
II. How to Use these Guidel<strong>in</strong>es ............................................................................................................ 3<br />
III. If You Are New to the <strong>DCED</strong> Standard .................................................................................................. 4<br />
IV. Key Pr<strong>in</strong>ciples to Implement<strong>in</strong>g the <strong>DCED</strong> Standard <strong>in</strong> CAEs ................................................................ 5<br />
1 Articulat<strong>in</strong>g Results Cha<strong>in</strong>s .................................................................................................................... 6<br />
1.1 Incorporat<strong>in</strong>g conflict <strong>in</strong>to results cha<strong>in</strong>s (Control Po<strong>in</strong>t 1.1-1.2) ................................................ 7<br />
1.2 Manag<strong>in</strong>g the results cha<strong>in</strong> (Control Po<strong>in</strong>ts 1.3 – 1.4) ................................................................ 18<br />
1.3 Key resources .............................................................................................................................. 19<br />
2 Def<strong>in</strong><strong>in</strong>g Indicators <strong>of</strong> Change ............................................................................................................. 20<br />
2.1 Def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators for results cha<strong>in</strong>s (Control Po<strong>in</strong>t 2.1) ........................................................... 21<br />
2.2 <strong>Measur<strong>in</strong>g</strong> the universal impact <strong>in</strong>dicators (Control Po<strong>in</strong>t 2.2) .................................................. 25<br />
2.3 Assess<strong>in</strong>g likelihood <strong>of</strong> susta<strong>in</strong>able impact (Control Po<strong>in</strong>t 2.3) .................................................. 26<br />
2.4 Manag<strong>in</strong>g <strong>in</strong>dicators (Control Po<strong>in</strong>t 2.4) ..................................................................................... 26<br />
2.5 Projections (Control Po<strong>in</strong>ts 2.5) .................................................................................................. 26<br />
2.6 Key Resources ............................................................................................................................. 27<br />
3 <strong>Measur<strong>in</strong>g</strong> Changes <strong>in</strong> Indicators ........................................................................................................ 27<br />
3.1 Collect<strong>in</strong>g basel<strong>in</strong>e <strong>in</strong>formation (Control Po<strong>in</strong>t 3.1) ................................................................... 28<br />
3.2 Good research practices (Control Po<strong>in</strong>t 3.2) ............................................................................... 29<br />
3.3 Qualitative <strong>in</strong>formation (Control Po<strong>in</strong>t 3.3) ................................................................................ 32<br />
3.4 Key resources .............................................................................................................................. 33<br />
4 Estimat<strong>in</strong>g Attributable Changes ........................................................................................................ 33<br />
4.1 A clear and appropriate system for attribution is <strong>in</strong> place (Control Po<strong>in</strong>t 4.1) .......................... 34<br />
4.2 Key resources .............................................................................................................................. 36<br />
5 Captur<strong>in</strong>g Wider Changes <strong>in</strong> the System or Market (Control Po<strong>in</strong>ts 5.1) ........................................... 36<br />
6 Track<strong>in</strong>g Programme Costs (Control Po<strong>in</strong>ts 6.1 – 6.2) ........................................................................ 37<br />
7 Report<strong>in</strong>g Results (Control Po<strong>in</strong>ts 7.1 – 7.5) ....................................................................................... 37<br />
8 Manag<strong>in</strong>g the System for Results Measurement ................................................................................ 38<br />
8.1 Establish<strong>in</strong>g a clear system for results measurement (Control Po<strong>in</strong>ts 8.1) ................................ 38<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
1
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
8.2 Human and f<strong>in</strong>ancial resources (Control Po<strong>in</strong>t 8.2) .................................................................... 39<br />
8.3 Key resources .............................................................................................................................. 40<br />
Annex 1: Key Def<strong>in</strong>itions ............................................................................................................................. 41<br />
Annex 2: Acronyms and Abbreviations .............................................................................................. 44<br />
Annex 3: Overall Resource List ............................................................................................................... 45<br />
List <strong>of</strong> Figures<br />
FIGURE 1: MAPPING COMPLEX CONFLICTS…………………………………………………………………………………………………………………. 7<br />
FIGURE 2: THE PSD-PEACEBUILDING CONTINUUM ............................................................................................................ 8<br />
FIGURE 3: MAPPING POSSIBLE NEGATIVE UNINTENTIONAL RESULTS .................................................................................... 13<br />
FIGURE 4: INDICATOR SELECTION TOOL…………………………………………………………………………………………………………………… 26<br />
FIGURE 5: FACTORS REPORTEDLY CONTRIBUTING TO IMPROVED FOOD SECURITY .................................................................... 35<br />
List <strong>of</strong> Tables<br />
TABLE 1: SIMPLIFIED EXAMPLE RESULTS CHAINS OF HOW PSD CAN CONTRIBUTE TO PEACEBUILDING…..…………………………………...15<br />
TABLE 2: CHALLENGING COMMON CONFLICT-RELATED ASSUMPTIONS ................................................................................... 17<br />
TABLE 3: DEFINING CONFLICT-SENSITIVE PSD INDICATORS .................................................................................................. 21<br />
TABLE 4: DEFINING INDICATORS TO TEST CONFLICT-RELEVANT RISKS ..................................................................................... 22<br />
TABLE 5: SAMPLE INDICATORS FOR MEASURING POSITIVE EFFECTS ON CONFLICT ..................................................................... 24<br />
Acknowledgements<br />
The authors wish to thank the many <strong>in</strong>terviewees who provided helpful guidance as we conducted our<br />
research. We are also grateful to the members <strong>of</strong> the Donor Committee for Enterprise <strong>Development</strong><br />
(<strong>DCED</strong>) for their support throughout, as well as Jim Tanburn and Mel<strong>in</strong>a He<strong>in</strong>rich from the <strong>DCED</strong><br />
Secretariat. We dedicate these guidel<strong>in</strong>es to the practitioners who are mak<strong>in</strong>g the effort to design and<br />
implement rigorous results measurement systems <strong>in</strong> conflict affected environments and dissem<strong>in</strong>ate<br />
their results to the rest <strong>of</strong> the field. These efforts are <strong>in</strong>valuable to advanc<strong>in</strong>g our knowledge <strong>in</strong> an area<br />
where much rema<strong>in</strong>s to be learned.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
2
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
I. Introduction<br />
This paper provides guidance for measur<strong>in</strong>g results <strong>in</strong> conflict affected environments (CAEs) us<strong>in</strong>g the<br />
Donor Committee for Enterprise <strong>Development</strong> (<strong>DCED</strong>) Standard for Results Measurement 1 . The <strong>DCED</strong><br />
Standard is a framework for measur<strong>in</strong>g and manag<strong>in</strong>g the results <strong>of</strong> private sector development<br />
programmes. The guidel<strong>in</strong>es follow the <strong>DCED</strong>’s def<strong>in</strong>ition <strong>of</strong> CAEs: “countries or regions where there is a<br />
high risk <strong>of</strong> violent conflict break<strong>in</strong>g out; that are <strong>in</strong> the midst <strong>of</strong> violent conflict; or have recently<br />
emerged from it, <strong>in</strong>clud<strong>in</strong>g countries classified as ‘post-conflict’”. 2 Def<strong>in</strong>itions for this and other<br />
commonly-used terms are provided <strong>in</strong> Annex 1, while acronyms and abbreviations are provided <strong>in</strong><br />
Annex 2.<br />
S<strong>in</strong>ce 2008, the <strong>DCED</strong> Standard has been implemented by numerous private sector development (PSD)<br />
projects across Africa and Asia. Feedback from practitioners <strong>in</strong>dicates that it helps monitor progress<br />
towards objectives and manage <strong>in</strong>terventions. In complex, rapidly chang<strong>in</strong>g environments it is<br />
particularly important for programmes to monitor, learn from and adapt their approach. 3 The emphasis<br />
<strong>of</strong> the <strong>DCED</strong> Standard on us<strong>in</strong>g monitor<strong>in</strong>g data to improve implementation is thus particularly valuable<br />
<strong>in</strong> CAEs, support<strong>in</strong>g the management <strong>of</strong> complex programmes and reduc<strong>in</strong>g the risk <strong>of</strong> caus<strong>in</strong>g harm<br />
through <strong>in</strong>appropriate <strong>in</strong>terventions. Nevertheless, the challenges <strong>of</strong> security, data availability, and<br />
staff<strong>in</strong>g have limited the application <strong>of</strong> the <strong>DCED</strong> Standard <strong>in</strong> CAEs to date.<br />
Donors are <strong>in</strong>creas<strong>in</strong>gly dedicat<strong>in</strong>g resources towards private sector development <strong>in</strong> CAEs, recognis<strong>in</strong>g<br />
the poor performance <strong>of</strong> CAEs towards meet<strong>in</strong>g development objectives such as the Millennium<br />
<strong>Development</strong> Goals. With greater fund<strong>in</strong>g, donors and implementers are be<strong>in</strong>g called upon to develop<br />
rigorous approaches for results measurement and report<strong>in</strong>g, <strong>in</strong> order to demonstrate impact and<br />
accountability. The <strong>DCED</strong> Standard is widely recognized as represent<strong>in</strong>g good practice <strong>in</strong> results<br />
measurement for PSD programm<strong>in</strong>g. Provid<strong>in</strong>g guidance <strong>in</strong> how to apply the Standard <strong>in</strong> CAEs can<br />
support practitioners to measure, manage and demonstrate results.<br />
II. How to Use these Guidel<strong>in</strong>es<br />
These guidel<strong>in</strong>es are written for donors, project implementers, auditors and consultants who are<br />
apply<strong>in</strong>g the <strong>DCED</strong> Standard <strong>in</strong> CAEs or consider<strong>in</strong>g its use. They are structured around the eight<br />
elements <strong>of</strong> the <strong>DCED</strong> Standard. For each component they note the challenges that affect application <strong>in</strong><br />
CAEs, provide guidance on mitigation strategies and refer to additional resources. Resources listed <strong>in</strong><br />
blue text l<strong>in</strong>k to a downloadable resource, and a full list <strong>of</strong> guidance documents are provided <strong>in</strong> Annex 3.<br />
Guidance is provided on how to structure a Standard-compliant results measurement system that, at a<br />
m<strong>in</strong>imum, avoids exacerbat<strong>in</strong>g conflict. For practitioners wish<strong>in</strong>g to use private sector development<br />
1 For more <strong>in</strong>formation about the <strong>DCED</strong> Standard, please refer to:<br />
http://www.enterprise-development.org/page/measur<strong>in</strong>g-and-report<strong>in</strong>g-results.<br />
2 The Donor Committee for Enterprise <strong>Development</strong>, <strong>Private</strong> sector development <strong>in</strong> conflict affected<br />
environments: Key Resources for Practitioners, 2010. The CAE def<strong>in</strong>ition is understood to also encompass the<br />
concept <strong>of</strong> fragility.<br />
3 Hummelbrunner, Richard and Jones, Harry, A guide for plann<strong>in</strong>g and strategy development <strong>in</strong> the face <strong>of</strong><br />
complexity, 2013, Overseas <strong>Development</strong> Institute.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
3
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
programmes as a means to build peace and promote stability, these guidel<strong>in</strong>es outl<strong>in</strong>e an approach for<br />
reflect<strong>in</strong>g this aim with<strong>in</strong> your results measurement system.<br />
In light <strong>of</strong> the great diversity among CAEs, readers should use these guidel<strong>in</strong>es as a resource <strong>in</strong><br />
develop<strong>in</strong>g and implement<strong>in</strong>g their monitor<strong>in</strong>g system, rather than a prescriptive set <strong>of</strong> rules. Many <strong>of</strong><br />
the recommendations are also relevant to PSD programm<strong>in</strong>g <strong>in</strong> humanitarian crises and other<br />
challeng<strong>in</strong>g contexts.<br />
These guidel<strong>in</strong>es assume that users have a basic knowledge <strong>of</strong> the <strong>DCED</strong> Standard and surround<strong>in</strong>g<br />
guidance, as described <strong>in</strong> the next section.<br />
III.<br />
If You Are New to the <strong>DCED</strong> Standard<br />
The <strong>DCED</strong> is a forum for bilateral donors and multilateral agencies seek<strong>in</strong>g to improve the effectiveness<br />
<strong>of</strong> their PSD programm<strong>in</strong>g.<br />
The <strong>DCED</strong> has published a Standard for <strong>Measur<strong>in</strong>g</strong> Results <strong>in</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong>, which<br />
provides a framework for programme managers to monitor their progress and report credible results. It<br />
can be implemented by the programme team and audited by an <strong>in</strong>dependent assessor to verify the<br />
credibility <strong>of</strong> the system. There are eight elements to the Standard, listed below. Follow the hyperl<strong>in</strong>ked<br />
bold text to download further guidance on each:<br />
1. Articulat<strong>in</strong>g Results Cha<strong>in</strong>s. Results cha<strong>in</strong>s visually represents how project activities will create<br />
outputs, outcomes and impact, show<strong>in</strong>g the causal l<strong>in</strong>ks and relationships between them. The<br />
results cha<strong>in</strong>s clearly demonstrate what the project is do<strong>in</strong>g and what changes are expected.<br />
2. Def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators <strong>of</strong> change. An <strong>in</strong>dicator specifies what you will measure <strong>in</strong> order to see whether<br />
change has occurred. By def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators, you clarify exactly how you will monitor your<br />
<strong>in</strong>terventions. Indicators are specified for each expected change outl<strong>in</strong>ed <strong>in</strong> results cha<strong>in</strong>s.<br />
3. <strong>Measur<strong>in</strong>g</strong> changes <strong>in</strong> <strong>in</strong>dicators. Once the <strong>in</strong>dicators have been def<strong>in</strong>ed, they are regularly<br />
monitored to see what has changed to help projects manage accord<strong>in</strong>gly.<br />
4. Estimat<strong>in</strong>g attributable changes. Once a change is observed, you need to estimate what can be<br />
attributed to your project. For example, an <strong>in</strong>crease <strong>in</strong> jobs may be due to your project – or because<br />
<strong>of</strong> the wider economic environment. Estimat<strong>in</strong>g attributable changes helps a project identify which<br />
<strong>in</strong>terventions are work<strong>in</strong>g and which are not.<br />
5. Captur<strong>in</strong>g wider changes <strong>in</strong> the system or market. Many PSD programmes aim to affect entire<br />
market systems. Monitor<strong>in</strong>g these changes gives a project a fuller picture <strong>of</strong> its impact, helps<br />
identify what is work<strong>in</strong>g and revise implementation strategies to maximize results.<br />
6. Track<strong>in</strong>g programme costs. In order to assess the success <strong>of</strong> the project it is necessary to know how<br />
much was spent to achieve the results.<br />
7. Report<strong>in</strong>g results. F<strong>in</strong>d<strong>in</strong>gs are communicated clearly to funders, local stakeholders, and to the<br />
wider development community where possible.<br />
8. Manag<strong>in</strong>g the system for results measurement. For a monitor<strong>in</strong>g and results measurement system<br />
to be effective, it must be adequately resourced and <strong>in</strong>tegrated <strong>in</strong>to all aspects <strong>of</strong> project<br />
management, <strong>in</strong>form<strong>in</strong>g the implementation and guid<strong>in</strong>g the strategy.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
4
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
The Standard divides each <strong>of</strong> these broad categories <strong>in</strong>to control po<strong>in</strong>ts and compliance criteria. The<br />
control po<strong>in</strong>ts summarise the exact requirements, while the compliance criteria describe how an auditor<br />
will assess the programme.<br />
Readers who are unfamiliar with the Standard can visit the <strong>DCED</strong>’s page on measur<strong>in</strong>g results to learn<br />
more about the <strong>DCED</strong> Standard. You can also download the Standard itself, or read the comprehensive<br />
implementation guidel<strong>in</strong>es downloadable through the above hyperl<strong>in</strong>ks or from the website here. It is<br />
advisable to review those documents before read<strong>in</strong>g these guidel<strong>in</strong>es.<br />
IV.<br />
Key Pr<strong>in</strong>ciples to Implement<strong>in</strong>g the <strong>DCED</strong> Standard <strong>in</strong> CAEs<br />
Three pr<strong>in</strong>ciples underlie the application <strong>of</strong> the <strong>DCED</strong> Standard <strong>in</strong> conflict affected areas: simplicity,<br />
flexibility and sensitivity.<br />
Simplicity. Personnel <strong>in</strong> CAEs are <strong>of</strong>ten overworked and under-resourced, with little tra<strong>in</strong><strong>in</strong>g <strong>in</strong> results<br />
measurement. High staff turnover may reduce familiarity with the project and context. An overly<br />
complex results measurement system will not be successfully implemented, wast<strong>in</strong>g project resources<br />
and reduc<strong>in</strong>g the will<strong>in</strong>gness <strong>of</strong> staff to use it.<br />
These guidel<strong>in</strong>es highlight techniques for simplify<strong>in</strong>g the results measurement system that enable it to<br />
be implemented even <strong>in</strong> challeng<strong>in</strong>g CAEs.<br />
Flexibility. CAEs are unpredictable, complicated, and fast-chang<strong>in</strong>g. An <strong>in</strong>flexible results measurement<br />
system may monitor irrelevant <strong>in</strong>dicators, fail to capture the positive impact <strong>of</strong> the project, and put staff<br />
and project clients at risk by ignor<strong>in</strong>g negative consequences.<br />
Consequently, your results measurement system should be flexible. Be aware that a results cha<strong>in</strong> can<br />
never capture the full complexity <strong>of</strong> the situation, and may become rapidly outdated if not regularly<br />
reviewed. Be alert for positive and negative changes not captured by your <strong>in</strong>dicators, and be prepared<br />
to modify the monitor<strong>in</strong>g system accord<strong>in</strong>gly. Aga<strong>in</strong>, these guidel<strong>in</strong>es suggest ways to manage this<br />
complexity while implement<strong>in</strong>g the <strong>DCED</strong> Standard.<br />
Sensitivity. Inappropriate <strong>in</strong>terventions <strong>in</strong> CAEs can endanger staff, partners and project clients while<br />
worsen<strong>in</strong>g the conflict. PSD projects must be particularly aware <strong>of</strong> potential negative impacts because<br />
conflicts are frequently driven by economic factors, which PSD <strong>in</strong>terventions can either re<strong>in</strong>force or<br />
reduce. For example, <strong>in</strong>creas<strong>in</strong>g the level <strong>of</strong> competition among firms can exacerbate conflict if one side<br />
perceives that their bus<strong>in</strong>esses are be<strong>in</strong>g displaced.<br />
Accord<strong>in</strong>gly, all projects work<strong>in</strong>g <strong>in</strong> CAEs should ’Do No Harm’. This means that they must understand<br />
the context <strong>in</strong> which they operate, understand the <strong>in</strong>teraction between their activities and the context,<br />
and act upon this understand<strong>in</strong>g to avoid negative impacts. 4 The results measurement system should<br />
thus monitor potential negative effects <strong>of</strong> the project on the conflict. Some programmes may wish to go<br />
4 Conflict Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peacebuild<strong>in</strong>g Resource Pack, 2004,<br />
3.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
5
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
further and explicitly aim to promote peace and stability through PSD. In this case, they may also<br />
monitor positive effects <strong>of</strong> the project on the conflict, although this poses greater challenges. Moreover,<br />
the monitor<strong>in</strong>g system must itself take the conflict <strong>in</strong>to account. Ask<strong>in</strong>g politically sensitive questions <strong>in</strong><br />
<strong>in</strong>terviews or shar<strong>in</strong>g identifiable <strong>in</strong>formation can put staff and beneficiaries at risk. These guidel<strong>in</strong>es<br />
highlight ways to <strong>in</strong>tegrate conflict sensitivity <strong>in</strong>to the results measurement system.<br />
Note that the follow<strong>in</strong>g sections will follow the sequence <strong>of</strong> the <strong>DCED</strong> Standard elements 1-8 and<br />
associated control po<strong>in</strong>ts.<br />
1 Articulat<strong>in</strong>g Results Cha<strong>in</strong>s<br />
Results cha<strong>in</strong>s visually articulate the activities, outputs, outcomes and impact <strong>of</strong> your project and the<br />
l<strong>in</strong>kages between them. This demonstrates what the project is do<strong>in</strong>g and what changes are expected as<br />
a result. If you have not worked with results cha<strong>in</strong>s before, you can download the <strong>DCED</strong> Results Cha<strong>in</strong><br />
guide here for a complete explanation.<br />
Results cha<strong>in</strong>s are important tools for results measurement <strong>in</strong> all projects, <strong>in</strong>clud<strong>in</strong>g those implemented<br />
<strong>in</strong> a CAE. The process <strong>of</strong> design<strong>in</strong>g results cha<strong>in</strong>s clarifies the logic <strong>of</strong> your project, forc<strong>in</strong>g you to be<br />
explicit about what changes you expect to see and how they relate to the desired outcomes. This allows<br />
your assumptions to be closely exam<strong>in</strong>ed and questioned, <strong>in</strong> order to check that the activities will<br />
plausibly lead to the planned results. Results cha<strong>in</strong>s also create the basis for results measurement. By<br />
articulat<strong>in</strong>g the logic <strong>of</strong> your <strong>in</strong>tervention, the results cha<strong>in</strong>s outl<strong>in</strong>e expected results. This allows you to<br />
regularly check whether your project is on track.<br />
Results Cha<strong>in</strong>s and Theories <strong>of</strong> Change<br />
The <strong>DCED</strong> Standard requires the use <strong>of</strong> results cha<strong>in</strong>s for each project <strong>in</strong>tervention, whereas the<br />
peacebuild<strong>in</strong>g community more frequently refers to theories <strong>of</strong> change. A theory <strong>of</strong> change is an<br />
approach to the design, monitor<strong>in</strong>g and evaluation <strong>of</strong> social programmes. It requires a clear articulation<br />
<strong>of</strong> the activities <strong>of</strong> the project and how it expects them to <strong>in</strong>teract with the context and key actors to<br />
lead to long-term change. This is referred to as the ‘logic’ <strong>of</strong> the project. It frequently <strong>in</strong>cludes a<br />
consideration <strong>of</strong> the context and key actors. Theories <strong>of</strong> change are sometimes depicted visually.<br />
Results cha<strong>in</strong>s are one way to visualise a theory <strong>of</strong> change that represents the series <strong>of</strong> specific changes<br />
that l<strong>in</strong>k project activities to impacts us<strong>in</strong>g a series <strong>of</strong> boxes and arrows. Whether you speak <strong>of</strong> results<br />
cha<strong>in</strong>s or theories <strong>of</strong> change, the key to their successful use is to cont<strong>in</strong>ually test the underly<strong>in</strong>g<br />
assumptions and use that to improve the project.<br />
There are several challenges to develop<strong>in</strong>g effective results cha<strong>in</strong>s <strong>in</strong> CAEs. These <strong>in</strong>clude:<br />
Knowledge gaps about market systems <strong>in</strong> CAEs. A good results cha<strong>in</strong> requires an <strong>in</strong>-depth<br />
knowledge <strong>of</strong> targeted market systems. For example, a PSD programme might seek to facilitate<br />
improvements <strong>in</strong> the quality <strong>of</strong> farmers’ produce, <strong>in</strong> order to <strong>in</strong>crease the prices that the farmer<br />
receives. This relies on the assumption that market pric<strong>in</strong>g is sensitive to product quality, which will<br />
not always be true. However, <strong>in</strong> a CAE it is <strong>of</strong>ten challeng<strong>in</strong>g to assess market systems, for a number<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
6
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
<strong>of</strong> reasons. There may be few people who understand market development <strong>in</strong> the local area, the<br />
market environment may have been rapidly<br />
chang<strong>in</strong>g as a result <strong>of</strong> the conflict, and it may be<br />
hard to gather new <strong>in</strong>formation due to a lack <strong>of</strong><br />
secondary <strong>in</strong>formation or formal <strong>in</strong>stitutions.<br />
Complexity. CAEs have multiple actors and<br />
numerous causes <strong>of</strong> conflict. This can result <strong>in</strong> a<br />
complex <strong>in</strong>teraction between the project and the<br />
context. Expected results have multiple causes, and<br />
project activities can have multiple effects – not all<br />
<strong>of</strong> which are l<strong>in</strong>ear or predictable. However, the<br />
results cha<strong>in</strong> design must balance complexity and<br />
usability. If it is overly simple, it will not capture the<br />
project logic. If it is overly complex, such as <strong>in</strong> the<br />
Figure 1: Mapp<strong>in</strong>g Complex Conflicts<br />
This excerpt from a diagram <strong>of</strong> the Afghan context<br />
shows how complex any attempt to fully represent a<br />
conflict can be.<br />
adjacent figure, 5 it is difficult to read and monitor aga<strong>in</strong>st. While this is true <strong>in</strong> every context, the<br />
presence <strong>of</strong> conflict <strong>in</strong>troduces additional complications that make this balanc<strong>in</strong>g act particularly<br />
challeng<strong>in</strong>g.<br />
Fast pace <strong>of</strong> change. Rapid change can render a results cha<strong>in</strong> out <strong>of</strong> date. For example,<br />
<strong>in</strong>tensification <strong>of</strong> a conflict may make market actors unwill<strong>in</strong>g to cont<strong>in</strong>ue participat<strong>in</strong>g <strong>in</strong> an<br />
<strong>in</strong>tervention and thus <strong>in</strong>validate that <strong>in</strong>tervention’s results cha<strong>in</strong>.<br />
Knowledge gaps about the relationship between PSD and peacebuild<strong>in</strong>g. While there are theories<br />
about the relationship between PSD <strong>in</strong>terventions and peacebuild<strong>in</strong>g, there is limited evidence on<br />
the detailed causal relationships, or the contexts <strong>in</strong> which those relationships hold. 6 For example, a<br />
recent systematic review by the Overseas <strong>Development</strong> Institute found that, out <strong>of</strong> 9,558 articles<br />
reviewed, only one provided any evidence regard<strong>in</strong>g whether or not employment creation<br />
promoted stability. 7<br />
1.1 Incorporat<strong>in</strong>g conflict <strong>in</strong>to results cha<strong>in</strong>s (Control Po<strong>in</strong>t 1.1-1.2)<br />
Develop<strong>in</strong>g and ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>g results cha<strong>in</strong>s requires staff to cont<strong>in</strong>ually exam<strong>in</strong>e their own assumptions<br />
and validate the l<strong>in</strong>kages between levels <strong>of</strong> their results cha<strong>in</strong>s. This is important <strong>in</strong> a CAE, where l<strong>in</strong>ks<br />
that are valid <strong>in</strong> a stable context may not hold. For example:<br />
Assumption: adequate trust exists for economic relationships to be built. Economic engagement<br />
between members <strong>of</strong> different groups requires trust. The United States Agency for International<br />
<strong>Development</strong> (USAID) found that ‘[b]ecause conflict breaks down trust between people, some forms<br />
<strong>of</strong> support and collaboration will only be possible when basic elements <strong>of</strong> trust are restored.’ 8<br />
5 Diagram cited <strong>in</strong> Banyan Global, Enterprise <strong>Development</strong>, Poverty and Conflict, 2010.<br />
6 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict & Economics: Lessons<br />
Learned on <strong>Measur<strong>in</strong>g</strong> Impact, 2011, 6.<br />
7 Holmes et al, What is the evidence on the impact <strong>of</strong> employment creation on stability and poverty reduction <strong>in</strong><br />
fragile states: A systematic review, 2013, v.<br />
8 USAID, Conflict and Livelihoods, 2005, 11.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
7
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Assumption: the private sector is will<strong>in</strong>g to engage. <strong>Private</strong> firms <strong>in</strong> CAEs may be less will<strong>in</strong>g to<br />
participate <strong>in</strong> PSD <strong>in</strong>itiatives than <strong>in</strong> other environments. Inter-group prejudices, security and other<br />
concerns can prevent the establishment <strong>of</strong> partnerships. The non-governmental organization (NGO)<br />
International Alert, for <strong>in</strong>stance, found that the private sector was “not overly enthusiastic” about<br />
provid<strong>in</strong>g employment opportunities to ex-combatants who had received vocational tra<strong>in</strong><strong>in</strong>g but<br />
who represented a potential security risk. 9 A long history <strong>of</strong> conflict may also have significantly<br />
deteriorated human capital as well; ex-combatants may not have basic skills required for<br />
employment (e.g. literacy).<br />
The results cha<strong>in</strong> design will depend on the project’s approach to address<strong>in</strong>g conflict. There are three<br />
broad approaches that PSD projects may use <strong>in</strong> CAEs:<br />
Conflict Bl<strong>in</strong>d. Projects do not <strong>in</strong>corporate the conflict <strong>in</strong>to their background research, design,<br />
implementation, or monitor<strong>in</strong>g.<br />
Do No Harm. Projects consciously look for and seek to avoid or mitigate negative impacts result<strong>in</strong>g<br />
from their activities. 10 Do<strong>in</strong>g No Harm requires the use <strong>of</strong> a conflict sensitive 11 approach.<br />
Peacebuild<strong>in</strong>g. Projects explicitly design their PSD <strong>in</strong>terventions to reduce conflict and build peace.<br />
A peacebuild<strong>in</strong>g project will aim to identify and address the underly<strong>in</strong>g cause(s) <strong>of</strong> conflict.<br />
Figure 2: The PSD-Peacebuild<strong>in</strong>g Cont<strong>in</strong>uum 12<br />
Focus on Conflict Resolution and Peacebuild<strong>in</strong>g<br />
<strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> Programm<strong>in</strong>g<br />
Conflict Bl<strong>in</strong>d Do No Harm Peacebuild<strong>in</strong>g<br />
Does not take conflict <strong>in</strong>to<br />
account <strong>in</strong> design,<br />
implementation, monitor<strong>in</strong>g,<br />
or evaluation<br />
Risks fail<strong>in</strong>g to meet<br />
objectives, and potentially<br />
caus<strong>in</strong>g harm<br />
Understands the conflict and<br />
conflict drivers<br />
Seeks to m<strong>in</strong>imize potential<br />
project harm by monitor<strong>in</strong>g<br />
un<strong>in</strong>tended adverse effects<br />
<strong>of</strong> project on conflict<br />
Results cha<strong>in</strong> articulates<br />
expected positive impacts <strong>of</strong><br />
project on reduc<strong>in</strong>g conflict<br />
Incorporates <strong>in</strong>dicators to<br />
test positive impacts <strong>of</strong><br />
project on conflict<br />
9 Godnick, William and Diana Kle<strong>in</strong>, The challenges <strong>of</strong> support<strong>in</strong>g ‘alternative’ economic opportunities for<br />
peacebuild<strong>in</strong>g – Perspectives from Colombia, 2009, 25.<br />
10 Interpeace, What is Peacebuild<strong>in</strong>g, 2010.<br />
11 A conflict sensitive project understands the context <strong>in</strong> which it works, comprehends the <strong>in</strong>teraction between its<br />
actions and the context and acts upon this understand<strong>in</strong>g <strong>in</strong> order to avoid negative impacts and maximise positive<br />
ones.<br />
12 Prepared by the authors, adapted from Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for<br />
projects <strong>in</strong> conflict affected countries, Phase 1: Formative research and key issue diagnosis, undated, 28.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
8
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
A “conflict bl<strong>in</strong>d” project may not achieve its objectives, due to unanticipated conflict-related factors, or<br />
<strong>in</strong>advertently worsen the conflict. The next two sections therefore explore how the project logic can<br />
<strong>in</strong>corporate Do No Harm and peacebuild<strong>in</strong>g pr<strong>in</strong>ciples.<br />
1.1.1 Do No Harm<br />
At a m<strong>in</strong>imum, projects <strong>in</strong> CAEs should aim<br />
to “Do No Harm”. They must be built on a<br />
comprehensive conflict analysis, and<br />
consider potential negative <strong>in</strong>teractions<br />
between the conflict and the context.<br />
1.1.1.1 Conflict Analysis<br />
“At the beg<strong>in</strong>n<strong>in</strong>g <strong>of</strong> the project, I was not<br />
conv<strong>in</strong>ced by peace and conflict analysis. I<br />
thought that I don’t have time for it. But<br />
I’ve discovered that, if I make the time, we<br />
avoid a lot <strong>of</strong> trouble. A good conflict<br />
analysis is vital to success.” – Liane Hryca,<br />
GIZ Afghanistan 13<br />
To be effective <strong>in</strong> a CAE, the project logic<br />
needs to be based on an understand<strong>in</strong>g <strong>of</strong><br />
the conflict. This should be done through a<br />
conflict analysis, which is a systematic<br />
study <strong>of</strong> the pr<strong>of</strong>ile, causes, actors, and dynamics <strong>of</strong> conflict. 14<br />
In particular, the conflict analysis should<br />
identify the drivers (i.e. underly<strong>in</strong>g causes) <strong>of</strong> conflict. For a selection <strong>of</strong> conflict analysis guidel<strong>in</strong>es, see<br />
the key resources section below.<br />
Key Questions for Conflict Analysis 15<br />
Conflict pr<strong>of</strong>ile<br />
What is the political, economic, and socio-cultural context<br />
What are the emergent political, economic, ecological and social issues<br />
What specific conflict prone/affected areas can be situated with<strong>in</strong> this context<br />
Is there a history <strong>of</strong> conflict<br />
Conflict causes<br />
What are structural causes <strong>of</strong> conflict<br />
What issues can be considered as proximate causes <strong>of</strong> conflict<br />
Impact <strong>of</strong> EU engagement <strong>in</strong> Afar Region, Ethiopia<br />
Afar is an <strong>in</strong>secure region <strong>in</strong> the East <strong>of</strong> Ethiopia, where<br />
regional <strong>in</strong>security, weak regional government, loss <strong>of</strong><br />
access to livelihood resources, and marg<strong>in</strong>alization <strong>of</strong><br />
m<strong>in</strong>ority groups, <strong>in</strong> particular pastoralist communities,<br />
all risk creat<strong>in</strong>g physical conflict.<br />
The EU provided approximately two billion Euros<br />
between 1976 and 1994 <strong>in</strong> aid to Ethiopia. In the Afar<br />
region, this <strong>in</strong>cluded large-scale irrigation projects on<br />
more than 40,000 ha <strong>of</strong> land.<br />
A report by Saferworld concluded that these projects<br />
have excluded pastoralists from plann<strong>in</strong>g and<br />
implementation, depriv<strong>in</strong>g them <strong>of</strong> graz<strong>in</strong>g land and<br />
<strong>in</strong>creas<strong>in</strong>g the risk <strong>of</strong> conflict between pastoralists,<br />
agriculturalists and the state.<br />
What triggers can contribute to the outbreak/further escalation <strong>of</strong> conflict<br />
13 Interview with Liane Hryca (GIZ Afghanistan), September 28, 2012.<br />
14 Conflict Sensitivity Consortium, How To Guide to Conflict Sensitivity, 2012.<br />
15 Conflict Sensitivity Consortium, How To Guide to Conflict Sensitivity, 2012.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
9
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Conflict actors<br />
Who are the ma<strong>in</strong> actors<br />
What are their ma<strong>in</strong> <strong>in</strong>terest, goals, positions, capacities, and relationships<br />
What <strong>in</strong>stitutional capacities for peace can be identified<br />
What actors can be identified as (potential) spoilers Why What are their <strong>in</strong>centives<br />
Conflict dynamics<br />
What are the current conflict trends<br />
What are w<strong>in</strong>dows <strong>of</strong> opportunity<br />
What scenarios can be developed from the analysis <strong>of</strong> the conflict pr<strong>of</strong>ile, causes and actors<br />
A conflict analysis for a PSD programme should pay particular attention to how economic factors divide<br />
and connect parties <strong>in</strong> a conflict, and exam<strong>in</strong>e the role played by economic actors. For example, <strong>in</strong> longterm<br />
conflicts warlords <strong>of</strong>ten effectively become bus<strong>in</strong>ess people, controll<strong>in</strong>g natural resources and<br />
provid<strong>in</strong>g employment. The conflict analysis should exam<strong>in</strong>e who benefits and loses economically from<br />
the conflict, and how external <strong>in</strong>tervention is likely to affect and be affected by the conflict. It should<br />
also exam<strong>in</strong>e the potential for the private sector to be a participant <strong>in</strong> the peace process, by <strong>of</strong>fer<strong>in</strong>g<br />
jobs to ex-combatants or support<strong>in</strong>g negotiations between different parties. 16<br />
The conflict analysis can be <strong>in</strong>corporated <strong>in</strong>to other analyses that PSD projects typically carry out to<br />
<strong>in</strong>form their design, such as market systems analysis or value cha<strong>in</strong> analysis. The <strong>DCED</strong> publication PSD<br />
<strong>in</strong> CAEs: Key Resources for Practitioners 17 describes seven tools that practitioners can use to design<br />
conflict-sensitive PSD programmes <strong>in</strong> CAEs. For example, GIZ <strong>in</strong> Sierra Leone has <strong>in</strong>corporated conflict<br />
sensitivity <strong>in</strong>to their selection <strong>of</strong> value cha<strong>in</strong>s. As well as evaluat<strong>in</strong>g economic potential, participatory<br />
workshops <strong>in</strong> each district explore the extent to which the value cha<strong>in</strong>s br<strong>in</strong>g different groups together,<br />
and the risks <strong>of</strong> foster<strong>in</strong>g <strong>in</strong>creased competition on conflict. 18<br />
F<strong>in</strong>d<strong>in</strong>gs from the conflict analysis will allow you to m<strong>in</strong>imise negative impacts on conflict and maximize<br />
positive ones by <strong>in</strong>corporat<strong>in</strong>g them <strong>in</strong>to the project design and results measurement system. This<br />
establishes an accurate programme logic, enables the development <strong>of</strong> conflict-sensitive <strong>in</strong>dicators, and<br />
ensures that changes <strong>in</strong> the context can be monitored and reflected <strong>in</strong> project design and the<br />
monitor<strong>in</strong>g system. This may be easier to achieve if staff conduct and analyse the conflict analysis,<br />
rather than outsourc<strong>in</strong>g it. 19 Ideally the <strong>in</strong>itial conflict analysis should be conducted prior to or dur<strong>in</strong>g the<br />
programme design phase, so that its f<strong>in</strong>d<strong>in</strong>gs can <strong>in</strong>form the programme. Critically, the conflict analysis<br />
must be regularly updated to detect any changes <strong>in</strong> the context and adjust <strong>in</strong>terventions that may<br />
exacerbate the conflict.<br />
16 One useful checklist is GTZ, Checklist: Economic Aspects <strong>of</strong> a Conflict<br />
17 <strong>DCED</strong>, <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflicted Affected Areas: Key Resources for Practitioners, 2010.<br />
18 Interview with Beatrice Tsch<strong>in</strong>kel (GIZ), September 17, 2012<br />
19 Conflict Sensitivity Consortium, How To Guide to Conflict Sensitivity, 2012, 5.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
10
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
1.1.1.2 Consider<strong>in</strong>g a project’s potential negative effects through results cha<strong>in</strong>s<br />
A PSD project can create or exacerbate conflict <strong>in</strong> various ways, <strong>in</strong>clud<strong>in</strong>g:<br />
Real or perceived bias <strong>in</strong> the distribution <strong>of</strong> project resources, support or employment<br />
Resource transfers to parties to the conflict<br />
Enabl<strong>in</strong>g a diversion <strong>of</strong> other resources to conflict (e.g. warr<strong>in</strong>g parties are freed from their<br />
obligation to support civilians)<br />
Contribut<strong>in</strong>g to <strong>in</strong>flation<br />
Chang<strong>in</strong>g exist<strong>in</strong>g power structures, for example by <strong>in</strong>creas<strong>in</strong>g competition <strong>in</strong> a market.<br />
Reduc<strong>in</strong>g the capacity <strong>of</strong> local structures. For example, recruit<strong>in</strong>g local government staff or activists<br />
(either formally or through payment <strong>of</strong> per diems for project work) or sett<strong>in</strong>g up parallel bodies. 20<br />
Consequently, it is essential for each project to consider its possible negative consequences. One wellestablished<br />
methodology for do<strong>in</strong>g this is the Do No Harm (DNH) approach. This is “an effort to identify<br />
the ways <strong>in</strong> which <strong>in</strong>ternational development assistance given <strong>in</strong> conflict sett<strong>in</strong>gs may be provided so<br />
that, rather than exacerbat<strong>in</strong>g and worsen<strong>in</strong>g the conflict, it helps local people disengage from fight<strong>in</strong>g<br />
and develop systems for settl<strong>in</strong>g the problems which prompt conflict with<strong>in</strong> their societies.” The<br />
methodology has been widely applied and has a large amount <strong>of</strong> guidance available. 21<br />
<strong>Private</strong> sector development programmes work<strong>in</strong>g <strong>in</strong> CAEs should adopt the DNH approach or a similar<br />
methodology to help them th<strong>in</strong>k about their effects on conflict. Several ways to <strong>in</strong>corporate the DNH<br />
approach <strong>in</strong> the creation <strong>of</strong> results cha<strong>in</strong>s are highlighted below.<br />
Case Study: Apply<strong>in</strong>g a Do No Harm Approach to PSD Interventions <strong>in</strong> Somaliland<br />
The USAID-funded Partnership for Economic Growth project, implemented by DAI <strong>in</strong> Somaliland, is<br />
focused on promot<strong>in</strong>g private sector development while us<strong>in</strong>g a Do No Harm approach. The project uses<br />
a multi-pronged approach to m<strong>in</strong>imize potential negative impacts <strong>of</strong> the project on clan-based tensions:<br />
1. Dur<strong>in</strong>g any market research and project design stages, the team conducts an ‘analysis <strong>of</strong> local political<br />
economy’. This maps potential scenarios <strong>in</strong> which project resources will <strong>in</strong>flame clan tensions, such as by<br />
<strong>in</strong>terven<strong>in</strong>g <strong>in</strong> value cha<strong>in</strong>s that are dom<strong>in</strong>ated by specific clans. This analysis allows the project to test<br />
that its <strong>in</strong>terventions will properly support competitiveness strategies while ensur<strong>in</strong>g clan balance.<br />
2. The project’s Communications and Community Outreach Officer and the security focal po<strong>in</strong>t<br />
cont<strong>in</strong>ually <strong>in</strong>form the project team <strong>of</strong> any possible land conflicts or related tensions <strong>in</strong> and around<br />
project <strong>in</strong>tervention areas and work closely with local government agencies and civil society<br />
stakeholders to ensure that all project activities are supported by neighbour<strong>in</strong>g communities.<br />
20 Anderson and Olson. Confront<strong>in</strong>g War: Critical Lessons for Peace Practitioners, 2003, 22-28.<br />
21 See http://www.cda<strong>in</strong>c.com/cdawww/project_pr<strong>of</strong>ile.phppid=DNH&pname=Do%20No%20Harm<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
11
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
3. The project will not directly support a bus<strong>in</strong>ess <strong>in</strong> a sector where there is a great deal <strong>of</strong> competition<br />
already (and therefore tensions may be exacerbated) or where donor support risks creat<strong>in</strong>g unfair<br />
advantages for bus<strong>in</strong>ess owners <strong>in</strong> one clan over those <strong>in</strong> another.<br />
4. Dur<strong>in</strong>g the project’s bus<strong>in</strong>ess match<strong>in</strong>g grants competition, all short-listed f<strong>in</strong>alists are requested to<br />
submit a “Bus<strong>in</strong>ess Risks and Assumptions Report” to be reviewed by the project team. With<strong>in</strong> this<br />
report, the grantee is required to lay out their “Do No Harm approach” dur<strong>in</strong>g the bus<strong>in</strong>ess expansion<br />
activity that the project will be support<strong>in</strong>g. Through site visits and follow up with key <strong>in</strong>dustry<br />
stakeholders, this report is verified before bus<strong>in</strong>ess match<strong>in</strong>g grants activities beg<strong>in</strong>.<br />
5. The project uses formal (e.g. through government) and <strong>in</strong>formal (e.g. between non-governmental<br />
organizations and local communities) communication approaches to expla<strong>in</strong> the importance <strong>of</strong> <strong>in</strong>direct<br />
benefits (e.g. new employment creation) aris<strong>in</strong>g from PSD <strong>in</strong>terventions and to therefore reduce<br />
pressure for direct project <strong>in</strong>terventions with all households.<br />
Identify conflict-related risks at each level <strong>of</strong> results cha<strong>in</strong>s<br />
Develop<strong>in</strong>g results cha<strong>in</strong>s can help th<strong>in</strong>k through the conflict-related risks at all levels, from activity to<br />
impact. While research<strong>in</strong>g these guidel<strong>in</strong>es, the authors found that practitioners tended to effectively<br />
consider short term risks caused by activities, such as conflict over distribution <strong>of</strong> resources. However, it<br />
is rarer for practitioners to consider how the outputs and outcomes <strong>of</strong> their projects might affect<br />
conflict. This step is essential <strong>in</strong> order to fully understand and manage the risks <strong>of</strong> the project.<br />
For example, risks created by the activities <strong>of</strong> a programme might <strong>in</strong>clude:<br />
Step <strong>in</strong> Results Cha<strong>in</strong><br />
Organisation provides entrepeneurship tra<strong>in</strong><strong>in</strong>g to<br />
participants<br />
Organisation identifies participants <strong>in</strong> tra<strong>in</strong><strong>in</strong>g<br />
session<br />
Conflict Related Risks<br />
Non-participants feel that the selection<br />
process was unfair.<br />
Potential participants compete over who<br />
gets selected.<br />
Local conflict parties require space <strong>in</strong><br />
tra<strong>in</strong><strong>in</strong>g for their own supporters.<br />
At the outcome level, risks might <strong>in</strong>clude:<br />
Step <strong>in</strong> Results Cha<strong>in</strong><br />
Participants start up their own<br />
bus<strong>in</strong>esses<br />
Participants <strong>in</strong>crease knowledge on how<br />
to start a bus<strong>in</strong>ess<br />
Conflict Related Risks<br />
New bus<strong>in</strong>esses conflict with exist<strong>in</strong>g<br />
bus<strong>in</strong>esses <strong>in</strong> same sector.<br />
New bus<strong>in</strong>esses pay taxes to local warlord,<br />
provid<strong>in</strong>g resources that fuel conflict.<br />
Participants can’t start bus<strong>in</strong>esses due to<br />
other constra<strong>in</strong>ts, and feel frustrated due<br />
to raised expectations.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
12
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Visually depict serious risks alongside results cha<strong>in</strong>s<br />
Serious risks can be represented more fully by creat<strong>in</strong>g a secondary strand <strong>in</strong> a results cha<strong>in</strong>. This will<br />
allow you to either adjust your activities to address the risk or – if you th<strong>in</strong>k the potential risk is small<br />
and the impact low – to assign <strong>in</strong>dicators to ensure that these risks are closely monitored. For example,<br />
the follow<strong>in</strong>g figure presents one example from East Timor. The project developed an agreement with<br />
the village community on the use <strong>of</strong> land and water, enabl<strong>in</strong>g young people to cultivate land and<br />
<strong>in</strong>tegrate <strong>in</strong>to communal life. This was expected to reduce the potential for conflict. However, the<br />
agreements would reduce the overall land available for each villager, potentially creat<strong>in</strong>g a new source<br />
<strong>of</strong> conflict, as identified on the left side <strong>of</strong> the results cha<strong>in</strong>. 22<br />
Figure 3:<br />
Mapp<strong>in</strong>g Possible Negative Un<strong>in</strong>tentional Results<br />
As serious risks are identified, it may be necessary to adjust exist<strong>in</strong>g activities or develop additional<br />
activities to mitigate the sources <strong>of</strong> conflict. These adjustments and their results should be <strong>in</strong>corporated<br />
dur<strong>in</strong>g subsequent updates to your results cha<strong>in</strong>s.<br />
Be aware that results cha<strong>in</strong>s <strong>of</strong>ten cannot capture all <strong>of</strong> the complex <strong>in</strong>teractions between conflict and<br />
your project. Be alert for unanticipated negative effects, and responsive to feedback from the field staff<br />
and community.<br />
22 Lange, Ralf et al.: Results and Results-based Monitor<strong>in</strong>g <strong>of</strong> the Contributions made by Vocational Education &<br />
Tra<strong>in</strong><strong>in</strong>g and the Promotion <strong>of</strong> Employment to Social (Re-)Integration <strong>in</strong> (Post-)Conflict Situations. Draft version –<br />
Unpublished, 2011, BMZ/GIZ, 11.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
13
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
1.1.1.3 Consider<strong>in</strong>g potential negative effects <strong>of</strong> conflict through results cha<strong>in</strong>s<br />
In addition to exam<strong>in</strong><strong>in</strong>g a project’s positive or negative impacts on conflict, it is also important to<br />
consider the conflict’s potential impact on project activities. These may <strong>in</strong>clude:<br />
Direct attacks on or threats to staff members or the project, limit<strong>in</strong>g their potential for engagement<br />
Beneficiaries chose not to participate <strong>in</strong> the project, because they do not see it as neutral or do not<br />
consider it safe to do so<br />
Inability to access project location, h<strong>in</strong>der<strong>in</strong>g implementation and monitor<strong>in</strong>g<br />
Project assets stolen by warr<strong>in</strong>g groups<br />
Initial project design rendered irrelevant by <strong>in</strong>tensification <strong>of</strong> conflict (e.g. project beneficiaries are<br />
forced to move)<br />
Such risks to project performance can be <strong>in</strong>corporated with<strong>in</strong> the risks and assumptions section <strong>of</strong> your<br />
project design documents. Once identified, they should <strong>in</strong>form the choice <strong>of</strong> project <strong>in</strong>terventions and<br />
strategies to mitigate their impacts.<br />
1.1.2 Incorporat<strong>in</strong>g Peacebuild<strong>in</strong>g <strong>in</strong>to PSD<br />
Conflict is <strong>of</strong>ten driven by economic factors 23 , and so PSD may play a critical role <strong>in</strong> creat<strong>in</strong>g the<br />
conditions for a stable society. It can address the causes or escalat<strong>in</strong>g factors <strong>of</strong> conflict, such as<br />
unemployment or poverty. It can also support peacebuild<strong>in</strong>g and reconciliation processes, for example<br />
by support<strong>in</strong>g the bus<strong>in</strong>ess community to negotiate between different parties to the conflict.<br />
Projects may consequently wish to build their programme logic around contribut<strong>in</strong>g to peacebuild<strong>in</strong>g.<br />
However, not all projects should try to capture their impact on conflict, as this requires adequate staff<br />
capacity, time, and fund<strong>in</strong>g. Few PSD projects have thus far been able to do so effectively. With<br />
sufficient resources, however, it is a very valuable project outcome to monitor.<br />
If projects did wish to capture their impact on stability, they should first outl<strong>in</strong>e their theory <strong>of</strong> change,<br />
or results cha<strong>in</strong>, expla<strong>in</strong><strong>in</strong>g how they expect to have a positive impact on the conflict. This should be<br />
rooted <strong>in</strong> the conflict analysis, and aim to address the identified drivers <strong>of</strong> conflict. To <strong>in</strong>form the<br />
development <strong>of</strong> peacebuild<strong>in</strong>g results cha<strong>in</strong>s, consider the follow<strong>in</strong>g list, which presents several <strong>of</strong> the<br />
most recognized ways <strong>in</strong> which PSD can contribute to peacebuild<strong>in</strong>g. 24 In practice, results cha<strong>in</strong>s are<br />
likely to be far more complex than these examples, with multiple levels and branches. For one example,<br />
see the <strong>DCED</strong> case study <strong>of</strong> GIZ <strong>in</strong> Sierra Leone.<br />
23 Love, Roy, Economic Drivers <strong>of</strong> Conflict and Cooperation <strong>in</strong> the Horn <strong>of</strong> Africa: A Regional Perspective and<br />
Overview, 2009.<br />
24 Drawn and adapted from The Donor Committee for Enterprise <strong>Development</strong>, <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong><br />
Conflict-Affected Environments – A Framework <strong>of</strong> Peacebuild<strong>in</strong>g Impacts, 2010; Rachel Goldwyn, Livelihoods and<br />
Peacebuild<strong>in</strong>g – what are current approaches to contribut<strong>in</strong>g to peacebuild<strong>in</strong>g through livelihoods <strong>in</strong>terventions<br />
(unpublished); Church, Cheyanne and Mark M. Roberts, Design<strong>in</strong>g for Results: Integrat<strong>in</strong>g M&E <strong>in</strong> Conflict<br />
Transformation Programmes, 2006; Mercy Corps. Evaluation and Assessment <strong>of</strong> Poverty and Conflict<br />
Interventions. Conflict & Economics: Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Undated.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
14
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Table 1: Simplified example results cha<strong>in</strong>s <strong>of</strong> how PSD can contribute to peacebuild<strong>in</strong>g<br />
Simplified Results Cha<strong>in</strong><br />
Higher levels<br />
<strong>of</strong> trust<br />
Reduced conflict<br />
Reduced levels <strong>of</strong> conflict<br />
Fewer young people out <strong>of</strong> work<br />
Improved economic opportunities<br />
for young people<br />
PSD <strong>in</strong>tervention<br />
Reduced conflict over resources<br />
Greater economic opportunities that<br />
do not rely on over-exploited<br />
resources<br />
PSD <strong>in</strong>tervention<br />
Increased economic<br />
<strong>in</strong>centives for peace<br />
Greater economic <strong>in</strong>teraction<br />
among parties to conflict<br />
PSD Intervention<br />
Explanation<br />
PSD <strong>in</strong>terventions foster greater economic <strong>in</strong>teraction among<br />
parties to a conflict, which creates economic <strong>in</strong>centives for<br />
peace and reduces the desire for conflict.<br />
Mercy Corps’ Build<strong>in</strong>g Bridges to Peace program <strong>in</strong> Uganda<br />
supported jo<strong>in</strong>t economic <strong>in</strong>itiatives such as farm<strong>in</strong>g and<br />
marketplace rehabilitation between groups that had<br />
previously clashed <strong>in</strong> the Northeast Karamoja region.<br />
Compared with communities that did not receive project<br />
activities, the targeted villages reported had better economic<br />
opportunities, reduced perceptions <strong>of</strong> <strong>in</strong>security and stronger<br />
<strong>in</strong>teraction between ethnic groups. 25<br />
PSD <strong>in</strong>terventions create <strong>in</strong>come and employment<br />
opportunities that reduce the <strong>in</strong>centives to engage <strong>in</strong> conflict.<br />
For <strong>in</strong>dividuals with no <strong>in</strong>come or jobs, conflict may be the<br />
only way to support themselves or their families. Even if they<br />
do not fight themselves, they may participate <strong>in</strong> war<br />
economies, such as by grow<strong>in</strong>g drugs. Employment<br />
opportunities <strong>in</strong> peaceful vocations reduce this <strong>in</strong>centive to go<br />
to war. This theory is <strong>of</strong>ten applied by ex-combatant<br />
re<strong>in</strong>tegration programmes. For example, a re<strong>in</strong>tegration<br />
programme <strong>in</strong> Colombia reviewed by International Alert 26<br />
<strong>of</strong>fered ex-combatants grants for livelihoods projects,<br />
employment, or further education. They reported that over<br />
14,300 ex-combatants had found employment.<br />
PSD <strong>in</strong>terventions <strong>in</strong>crease economic opportunities that do<br />
not rely on over-exploited resources, thereby reduc<strong>in</strong>g a key<br />
driver <strong>of</strong> conflict.<br />
Conflicts are <strong>of</strong>ten driven by <strong>in</strong>adequate access to resources<br />
that are critical to livelihoods (e.g. water, graz<strong>in</strong>g land).<br />
Negotiated agreements over the use <strong>of</strong> such resources are<br />
most susta<strong>in</strong>able where groups have other economic<br />
opportunities. In Somalia, Mercy Corps supported the<br />
negotiation <strong>of</strong> eight agreements to limit competition for scarce<br />
resources <strong>in</strong>clud<strong>in</strong>g trees while simultaneously support<strong>in</strong>g<br />
alternative economic opportunities such as nursery<br />
management. These alternative livelihood opportunities can<br />
reduce the prevalence <strong>of</strong> economic activities that put a stra<strong>in</strong><br />
on over-exploited resources (e.g. charcoal production). 27<br />
25 Mercy Corps, Peacebuild<strong>in</strong>g through Economic <strong>Development</strong> Approach, 2011, 2.<br />
26 Godnick and Kle<strong>in</strong>, The challenges <strong>of</strong> support<strong>in</strong>g ‘Alternative’ economic opportunities for peacebuild<strong>in</strong>g –<br />
perspectives from Colombia, 2009, 24<br />
27 Mercy Corps, Peacebuild<strong>in</strong>g through Economic <strong>Development</strong> Approach, 2011, 3.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
15
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Simplified Results Cha<strong>in</strong><br />
Reduced<br />
diversion <strong>of</strong><br />
resources to<br />
parties to the<br />
conflict<br />
Reduced conflict<br />
PSD Intervention<br />
Reduced conflict<br />
Reduced tension between<br />
different groups<br />
Reduced <strong>in</strong>equality<br />
Increased flow <strong>of</strong><br />
resources to<br />
government (e.g.<br />
through tax)<br />
<strong>Private</strong> sector improves governance<br />
Explanation<br />
PSD <strong>in</strong>terventions improve the governance <strong>of</strong> the private<br />
sector, which <strong>in</strong> turn puts pressure on the state to improve its<br />
own governance and support conflict reduction. As a major<br />
societal actor, the decisions and behaviour <strong>of</strong> the private<br />
sector has a major role <strong>in</strong> <strong>in</strong>fluenc<strong>in</strong>g government behaviour<br />
and actions. Initiatives such as the Extractive Industries<br />
Transparency Initiative encourage private firms to behave<br />
responsibly and transparently <strong>in</strong> their relations with the state,<br />
report<strong>in</strong>g what payments are made and to whom. This can<br />
reduce conflict both directly, by deny<strong>in</strong>g combatants the funds<br />
needed to fuel conflict, and <strong>in</strong>directly, as the private sector<br />
<strong>in</strong>creases use <strong>of</strong> formal state channels and processes – rather<br />
than circumvent<strong>in</strong>g these processes through pay<strong>in</strong>g bribes. A<br />
stronger and better resourced government is <strong>in</strong> a stronger<br />
position to reduce conflict. However, <strong>in</strong> many contexts the<br />
government is also a cause <strong>of</strong> context – so this assumption (as<br />
with the others listed here) would have to be validated<br />
through a conflict analysis.<br />
PSD <strong>in</strong>terventions reduce <strong>in</strong>equality between parties to a<br />
conflict, thus reduc<strong>in</strong>g resentment and the likelihood <strong>of</strong><br />
conflict. Where conflict is driven by resentment and anger<br />
over <strong>in</strong>equitable distribution <strong>of</strong> resources, PSD <strong>in</strong>terventions<br />
that support economic improvements by marg<strong>in</strong>alized<br />
populations may reduce these tensions and thus support<br />
peace.<br />
PSD <strong>in</strong>tervention<br />
Each <strong>of</strong> the above theories is based on assumptions about the underly<strong>in</strong>g causes <strong>of</strong> conflict, which will<br />
not hold true <strong>in</strong> every situation. The logic <strong>of</strong> the results cha<strong>in</strong> should be <strong>in</strong>formed by a rigorous conflict<br />
analysis that identifies the drivers <strong>of</strong> conflict and analyses how the project can best address them.<br />
Results cha<strong>in</strong>s will make these assumptions explicit. PSD <strong>in</strong>terventions do not automatically have a<br />
positive impact on a conflict, 28 so your monitor<strong>in</strong>g system should research and test them. The table<br />
below presents common parts <strong>of</strong> peacebuild<strong>in</strong>g results cha<strong>in</strong>s and exam<strong>in</strong>es the assumptions underly<strong>in</strong>g<br />
them.<br />
28 Gündüz, Canan and Diane Kle<strong>in</strong>, Conflict Sensitive Approaches To Value Cha<strong>in</strong> <strong>Development</strong>, 2008, 21.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
16
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Table 1:<br />
Challeng<strong>in</strong>g Common Conflict-Related Assumptions<br />
Step <strong>in</strong> Results Cha<strong>in</strong><br />
Increased trust<br />
between conflict<strong>in</strong>g<br />
groups<br />
Increased economic<br />
<strong>in</strong>teraction betwen<br />
conflict<strong>in</strong>g groups<br />
Reduced <strong>in</strong>centive to<br />
engage <strong>in</strong> conflict<br />
Increased<br />
employment/<strong>in</strong>come<br />
Reduced<br />
conflict<br />
Reduced<br />
<strong>in</strong>equality<br />
Assumptions<br />
Economic <strong>in</strong>teraction will build trust. Economic <strong>in</strong>teractions may<br />
reduce rather than <strong>in</strong>crease levels <strong>of</strong> trust where they <strong>in</strong>crease<br />
competition. For example, a jo<strong>in</strong>t marketplace could have a negative<br />
impact if different groups are compet<strong>in</strong>g for sales. It is likely that some<br />
k<strong>in</strong>ds <strong>of</strong> economic relationships contribute more strongly than others<br />
to build<strong>in</strong>g trust. Mercy Corps observed that “‘Deep’ economic<br />
<strong>in</strong>teractions (such as participation <strong>in</strong> economic associations or bus<strong>in</strong>ess<br />
partnerships) may build stronger relationships between adversarial<br />
groups and provide a stronger <strong>in</strong>centive for peace than ‘th<strong>in</strong>’ economic<br />
<strong>in</strong>teractions (such as trad<strong>in</strong>g at a local market).” 29<br />
Conflict is primarily driven by desire for additional resources. Conflict<br />
frequently has multiple causes, <strong>in</strong>clud<strong>in</strong>g ideology, discrim<strong>in</strong>ation, and<br />
<strong>in</strong>equality. The causes will vary from country to country. The World<br />
Bank found that, across seven countries, 40% <strong>of</strong> rebels jo<strong>in</strong> because <strong>of</strong><br />
unemployment or idleness – while only 13% jo<strong>in</strong> because <strong>of</strong> belief <strong>in</strong><br />
the cause. 30 But <strong>in</strong> Mali and the West Bank the trend was reversed,<br />
with 46% cit<strong>in</strong>g belief <strong>in</strong> the cause, and about 20% cit<strong>in</strong>g<br />
unemployment or idleness. Consequently, this theory <strong>of</strong> change would<br />
likely be less appropriate <strong>in</strong> the latter two areas.<br />
Lower levels <strong>of</strong> <strong>in</strong>equality reduce the risk <strong>of</strong> conflict. While overall<br />
<strong>in</strong>equality <strong>in</strong> the economy is not associated with a higher risk <strong>of</strong><br />
conflict, case study evidence suggests that horizontal <strong>in</strong>equality<br />
between regions or ethnic groups is. 31 Horizontal <strong>in</strong>equality <strong>of</strong>ten<br />
correlates with overall <strong>in</strong>equality, but not always; a society might have<br />
high levels <strong>of</strong> <strong>in</strong>equality between ethnic groups, but low overall levels<br />
<strong>of</strong> <strong>in</strong>equality. If a PSD project seeks to reduce conflict by reduc<strong>in</strong>g<br />
<strong>in</strong>equality, it should clarify <strong>in</strong> its results cha<strong>in</strong> which groups it expects<br />
to work with, and how <strong>in</strong>equality between these groups causes<br />
conflict.<br />
Project learn<strong>in</strong>g and adaptation is a critical to develop<strong>in</strong>g effective results cha<strong>in</strong>s <strong>in</strong> CAEs. The follow<strong>in</strong>g<br />
case study <strong>of</strong> the International Rescue Committee’s project <strong>in</strong> Liberia emphasizes the importance <strong>of</strong><br />
collect<strong>in</strong>g <strong>in</strong>formation to test a project’s assumptions about the drivers <strong>of</strong> conflict. 32<br />
29 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions Conflict & Economics: Lessons<br />
Learned on <strong>Measur<strong>in</strong>g</strong> Impact, undated, 5.<br />
30 World Bank, World <strong>Development</strong> Report 2011: Conflict, Security and <strong>Development</strong>, 2011, 80.<br />
31 Humphreys, Macartan, Economies and Violent Conflict, 2002, 3-4.<br />
32 Fearon, James et al, Evaluat<strong>in</strong>g Community-Driven Reconstruction: Lessons from post-conflict Liberia.<br />
<strong>Development</strong> Outreach, 2009.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
17
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Case Study: Test<strong>in</strong>g Assumptions <strong>in</strong> Liberia<br />
International Rescue Committee’s Community Driven Reconstruction project sought to <strong>in</strong>crease social<br />
cohesion with<strong>in</strong> villages <strong>in</strong> Liberia, on the assumption that this had been destroyed by many years <strong>of</strong><br />
civil war. The logic <strong>of</strong> one <strong>of</strong> their project’s <strong>in</strong>terventions – provid<strong>in</strong>g fund<strong>in</strong>g for committees to spend<br />
on community projects – is illustrated by the simplified results cha<strong>in</strong> below. The results cha<strong>in</strong> shows<br />
the project assumed their <strong>in</strong>terventions would lead to <strong>in</strong>creased cohesion, improved attitudes to<br />
governance, and improved participation <strong>of</strong> m<strong>in</strong>ority groups. However, although the <strong>in</strong>tervention did<br />
<strong>in</strong>crease levels <strong>of</strong> cohesion, democratic attitudes and social acceptance, the basel<strong>in</strong>e discovered that<br />
these were high with<strong>in</strong> treatment and control communities alike before the start <strong>of</strong> the project. This<br />
challenged the underly<strong>in</strong>g assumption <strong>of</strong> the project that years <strong>of</strong> civil war had significantly reduced<br />
the social cohesion with<strong>in</strong> village. The evaluation found it plausible that “the really important divisions<br />
exist between communities rather than with<strong>in</strong> them”. This f<strong>in</strong>d<strong>in</strong>g underscores the importance <strong>of</strong><br />
test<strong>in</strong>g important assumptions prior to and dur<strong>in</strong>g implementation and reflect<strong>in</strong>g this learn<strong>in</strong>g <strong>in</strong><br />
project <strong>in</strong>terventions <strong>in</strong> order to ensure that they have their desired effects.<br />
1.2 Manag<strong>in</strong>g the results cha<strong>in</strong> (Control Po<strong>in</strong>ts 1.3 – 1.4)<br />
In order to successfully manage results cha<strong>in</strong>s <strong>in</strong> a CAE:<br />
Keep it simple. With high staff turnover and limited resources, there is typically limited capacity to<br />
use and manage results cha<strong>in</strong>s. Consider reduc<strong>in</strong>g the number <strong>of</strong> steps to show only key changes<br />
that are expected, or splitt<strong>in</strong>g a bigger <strong>in</strong>tervention <strong>in</strong>to smaller, more manageable results cha<strong>in</strong>s.<br />
Adapt the complexity to the capacity <strong>of</strong> the team.<br />
Regularly review and revise (as needed). There is a risk that the use <strong>of</strong> results cha<strong>in</strong>s can lead<br />
project staff to adopt an <strong>in</strong>flexible, l<strong>in</strong>ear approach. This is <strong>in</strong>appropriate <strong>in</strong> CAEs, where project staff<br />
need the flexibility to respond to unpredictable situations. The results cha<strong>in</strong>s should be reviewed<br />
regularly (at least every six months), and adjusted when circumstances change or the orig<strong>in</strong>al results<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
18
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
cha<strong>in</strong>s are <strong>in</strong>validated. 33 They should not be considered a prescriptive guide to programme activities,<br />
but a way to explore and challenge the assumptions <strong>of</strong> the programme.<br />
Consult all stakeholders. This is particularly important <strong>in</strong> a conflict situation, where you need<br />
acceptance from multiple groups. Stakeholder consultations can themselves have a positive impact<br />
upon the conflict by creat<strong>in</strong>g opportunities for shar<strong>in</strong>g and jo<strong>in</strong>t ownership. Consider whether<br />
groups <strong>in</strong> conflict should be consulted jo<strong>in</strong>tly or separately. However, be aware that groups may be<br />
biased towards particular positions, and consultations should be carefully planned so as not to<br />
favour any particular party. GTZ notes that “particularly <strong>in</strong> direct post-conflict situations,<br />
organisations and <strong>in</strong>stitutions with direct l<strong>in</strong>ks to conflict parties are <strong>of</strong>ten the most visible but not<br />
necessarily the most eligible partners.” 34 This is an extremely difficult issue for many projects <strong>in</strong><br />
CAEs, and it is essential to take the time to understand the different group dynamics and consult<br />
widely.<br />
Support project staff to understand the results cha<strong>in</strong>s and use it to drive their decisions. In a CAE<br />
various factors can m<strong>in</strong>imize staff understand<strong>in</strong>g <strong>of</strong> results cha<strong>in</strong>s, <strong>in</strong>clud<strong>in</strong>g high staff turnover,<br />
<strong>in</strong>creased use <strong>of</strong> partners, and low staff capacity. Consider mak<strong>in</strong>g it part <strong>of</strong> the <strong>in</strong>duction for new<br />
staff, and hold<strong>in</strong>g regular tra<strong>in</strong><strong>in</strong>g on it. For more strategies to address human resources issues, see<br />
section 8 on manag<strong>in</strong>g the results measurement system.<br />
1.3 Key resources<br />
1.3.1 Conflict Analysis Guidel<strong>in</strong>es<br />
Conflict Sensitivity Consortium. Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian<br />
Assistance and Peace Build<strong>in</strong>g Resource Pack, Chapter Two. Very comprehensive conflict sensitivity<br />
guidel<strong>in</strong>es. Chapter two describes conflict analysis <strong>in</strong> detail and summarises 15 different conflict analysis<br />
tools from major donors. For each they discuss the primary purpose, summarise the ma<strong>in</strong> steps, and<br />
give examples <strong>of</strong> lessons learned and current applications. If you are select<strong>in</strong>g a conflict analysis<br />
framework, this is an <strong>in</strong>valuable resource.<br />
Collaborative for <strong>Development</strong> Action, Inc. The Do No Harm Handbook: The Framework for Analyz<strong>in</strong>g<br />
the Impact <strong>of</strong> Assistance on Conflict. This guide provides a step-by-step guide to assess<strong>in</strong>g the potential<br />
<strong>of</strong> a project to cause harm <strong>in</strong> its selected context.<br />
GIZ. Peace and Conflict Assessment (PCA): A Methodological Framework for the Conflict- and Peace-<br />
Oriented Alignment <strong>of</strong> <strong>Development</strong> Programmes. Exam<strong>in</strong>es the application <strong>of</strong> peace and conflict<br />
assessments at multiple stages <strong>in</strong> a project, <strong>in</strong>clud<strong>in</strong>g impact monitor<strong>in</strong>g.<br />
USAID. Conflict Assessment Framework. Version 2.0. Provides a framework for diagnos<strong>in</strong>g a conflict<br />
and generat<strong>in</strong>g responses. Application guidel<strong>in</strong>es can be found here.<br />
1.3.2 Results Cha<strong>in</strong>s and Theories <strong>of</strong> Change<br />
Becker, Sab<strong>in</strong>e. Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes. Reviews<br />
many <strong>of</strong> the key relationships between PSD and peacebuild<strong>in</strong>g.<br />
33 Vogel, Isabel, Review <strong>of</strong> the use <strong>of</strong> ‘Theory Of Change’ In <strong>in</strong>ternational development’, 2012, 27.<br />
34 Becker, Sab<strong>in</strong>e, Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes, 2006, GTZ<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
19
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
CARE International UK. Peacebuild<strong>in</strong>g with Impact: Def<strong>in</strong><strong>in</strong>g Theories <strong>of</strong> Change. Offers guidance <strong>in</strong><br />
develop<strong>in</strong>g the logic <strong>of</strong> how project <strong>in</strong>terventions will impact upon peacebuild<strong>in</strong>g.<br />
Davis, Peter. Corporations, Global Governance and Post-Conflict Reconstruction. Synthesizes the<br />
impacts corporations may have on conflict and reconstruction, draw<strong>in</strong>g from Azerbaijan, Bosnia and<br />
Rwanda.<br />
Humphreys, Macartan. Economics and Violent Conflict. Reviews the evidence for l<strong>in</strong>kages between<br />
economics and conflict.<br />
Mercy Corps. Peacebuild<strong>in</strong>g through Economic <strong>Development</strong> Approach. Outl<strong>in</strong>es five theories <strong>of</strong><br />
change that Mercy Corps is implement<strong>in</strong>g and test<strong>in</strong>g through its work and provides examples <strong>of</strong> each.<br />
Mierke, Axel. Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes. Analyses the<br />
potential contribution <strong>of</strong> PSD programm<strong>in</strong>g to peacebuild<strong>in</strong>g.<br />
Pottebaum, David. Relationships between Conflict, Poverty, Inequality, and Economic Growth.<br />
Reviews the evidence for relationships between conflict, poverty, <strong>in</strong>equality, and economic growth.<br />
Sen, Nabanita. A Guide to Mak<strong>in</strong>g Results Cha<strong>in</strong>s. Provides the key steps required to create a results<br />
cha<strong>in</strong>.<br />
USAID. Conflict-Sensitive Approaches to Value Cha<strong>in</strong> <strong>Development</strong>. Outl<strong>in</strong>es how to <strong>in</strong>corporate a<br />
conflict lens when conduct<strong>in</strong>g value cha<strong>in</strong>s analysis. The f<strong>in</strong>d<strong>in</strong>gs can <strong>in</strong>form the logic <strong>of</strong> a project’s<br />
results cha<strong>in</strong>s.<br />
Vogel, Isabel. Review <strong>of</strong> the uses <strong>of</strong> ‘Theory <strong>of</strong> Change’ <strong>in</strong> International <strong>Development</strong>. Exam<strong>in</strong>es how<br />
theories <strong>of</strong> change are used <strong>in</strong> the field, their different applications and best practices.<br />
1.3.3 Conflict Sensitivity<br />
Conflict Sensitivity Consortium. Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian<br />
Assistance and Peace Build<strong>in</strong>g Resource Pack. Provides guidance on how to <strong>in</strong>corporate a conflict<br />
sensitive approach throughout the project cycle, <strong>in</strong>clud<strong>in</strong>g monitor<strong>in</strong>g and evaluation.<br />
Conflict Sensitivity Consortium. A How To Guide to Conflict Sensitivity. A more concise and updated<br />
version <strong>of</strong> the above document.<br />
2 Def<strong>in</strong><strong>in</strong>g Indicators <strong>of</strong> Change<br />
A results measurement system should only track what you are attempt<strong>in</strong>g to <strong>in</strong>fluence through your<br />
work, or factors that will have a strong <strong>in</strong>fluence on your work. This ensures that you do not spend time<br />
and money measur<strong>in</strong>g irrelevant factors, which is particularly important <strong>in</strong> a CAE, due to the common<br />
constra<strong>in</strong>ts <strong>of</strong> money and staff time. The <strong>DCED</strong> Standard requires <strong>in</strong>dicators to be l<strong>in</strong>ked to the results<br />
cha<strong>in</strong>, <strong>in</strong> order to ensure that they are relevant to the project.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
20
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
A pr<strong>in</strong>ciple challenge to def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators <strong>in</strong> CAEs is that relevant conflict-relevant <strong>in</strong>dicators are still<br />
be<strong>in</strong>g understood globally – particularly at the impact level – and their applicability across contexts is<br />
unclear. 35<br />
2.1 Def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators for results cha<strong>in</strong>s (Control Po<strong>in</strong>t 2.1)<br />
The <strong>DCED</strong> Standard requires the track<strong>in</strong>g <strong>of</strong> each change <strong>in</strong> a results cha<strong>in</strong>. Projects need to carefully<br />
assess the changes they must measure to assess the theory <strong>of</strong> change, recogniz<strong>in</strong>g that capacity and<br />
resources are typically very limited <strong>in</strong> a CAE.<br />
PSD projects <strong>in</strong> CAEs should disaggregate key <strong>in</strong>dicators by parties to the conflict, and <strong>in</strong>corporate<br />
additional <strong>in</strong>dicators that would not be necessary <strong>in</strong> stable environments. As <strong>in</strong>troduced <strong>in</strong> Section 1, all<br />
projects should set <strong>in</strong>dicators to monitor the negative effects <strong>of</strong> their work, and ensure that they are not<br />
<strong>in</strong>advertently caus<strong>in</strong>g harm. Projects with a peacebuild<strong>in</strong>g objective may also wish to set <strong>in</strong>dicators to<br />
exam<strong>in</strong>e the positive impact <strong>of</strong> their work.<br />
2.1.1 Disaggregate key changes by parties to the conflict<br />
Projects should disaggregate key <strong>in</strong>dicators by parties to the conflict, <strong>in</strong> order to:<br />
Ensure that the project is not disproportionately benefit<strong>in</strong>g one group, which might create conflict.<br />
Reveal potential positive impacts on conflict, such as <strong>in</strong>creased ethnic diversity among partners’<br />
workforces.<br />
“If you ask someone about their clan <strong>in</strong> Hargeisa (Somaliland), you will be put on the first plane out <strong>of</strong><br />
there.” – Interviewee, SEED Somalia<br />
While disaggregat<strong>in</strong>g <strong>in</strong>dicators by ethnic group can be helpful, it can also add to tensions. For <strong>in</strong>stance,<br />
<strong>in</strong> Somalia it is generally taboo to ask about clan affiliation. Measurement by geographic area may<br />
therefore be more appropriate where this serves as a proxy for ethnicity. 36 The table below shows some<br />
examples <strong>of</strong> disaggregat<strong>in</strong>g PSD <strong>in</strong>dicators:<br />
Table 2:<br />
Def<strong>in</strong><strong>in</strong>g Conflict-Sensitive PSD Indicators<br />
Standard PSD Indicator<br />
Jobs created as a result <strong>of</strong><br />
programme activities<br />
Change <strong>in</strong> SME net <strong>in</strong>come<br />
Disaggregated PSD Indicator<br />
Jobs created as a result <strong>of</strong> programme activities (disaggregated by<br />
relevant party to the conflict / geographic region)<br />
Change <strong>in</strong> SME net <strong>in</strong>come (disaggregated by relevant party to the<br />
conflict / geographic region)<br />
2.1.2 Assess<strong>in</strong>g negative effects <strong>of</strong> the project on conflict<br />
All projects should use their results measurement system to ensure they avoid conflict. There are<br />
various ways <strong>in</strong> which a PSD project can create or exacerbate conflict, as noted above <strong>in</strong> Section 1.<br />
35 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict & Economics: Lessons<br />
Learned on <strong>Measur<strong>in</strong>g</strong> Impact, undated, 6.<br />
36 ECOPA, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict-affected countries, Phase 2: Case studies and<br />
recommendations, August 2011, 41-42.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
21
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Indicators should be tailored to the identified conflict risks and l<strong>in</strong>ked to project activities and expected<br />
outcomes. They must be <strong>in</strong>formed by the conflict assessment, as different conflicts will have different<br />
observable effects. 37 Indicators can act as early warn<strong>in</strong>g signs <strong>of</strong> conflict and enable programme staff to<br />
understand the long-term effects <strong>of</strong> the programme and manage it accord<strong>in</strong>gly.<br />
At the activity level, simple <strong>in</strong>dicators can serve as early warn<strong>in</strong>g systems <strong>of</strong> the conflict. For example,<br />
the SEED programme <strong>in</strong> Somalia used the follow<strong>in</strong>g <strong>in</strong>dicators:<br />
Boycott <strong>of</strong> programme meet<strong>in</strong>gs by <strong>in</strong>vited parties<br />
Un<strong>of</strong>ficial meet<strong>in</strong>gs with a negative agenda convened locally to discuss the programme<br />
Reluctance by state <strong>of</strong>ficials, <strong>in</strong>terest groups, or programme clients to participate <strong>in</strong> programme<br />
<strong>in</strong>terventions<br />
Compla<strong>in</strong>ts from state <strong>of</strong>ficials or community leaders <strong>of</strong> non-<strong>in</strong>volvement/non-recognition<br />
Segregation <strong>of</strong> certa<strong>in</strong> organized <strong>in</strong>terest groups/<strong>in</strong>dividuals <strong>in</strong> community mobilization<br />
processes<br />
Lack <strong>of</strong> participation by stakeholders <strong>in</strong> project <strong>in</strong>terventions<br />
Threats to staff/partners<br />
Expression <strong>of</strong> grievances or discontent around resource distribution<br />
Use <strong>of</strong> political, social or economic <strong>in</strong>fluence to <strong>in</strong>terfere with programme <strong>in</strong>terventions<br />
For more <strong>in</strong>dicator examples and details on their use, see the full case study <strong>of</strong> the SEED programme on<br />
the <strong>DCED</strong> website.<br />
The close monitor<strong>in</strong>g <strong>of</strong> such <strong>in</strong>dicators – as well as vigilant observation by programme staff <strong>of</strong><br />
unexpected or unusual behaviours – can provide early warn<strong>in</strong>g to a project <strong>of</strong> potential dangers.<br />
Nevertheless, recognize that a project can never anticipate all potential sources <strong>of</strong> conflict and your<br />
conflict <strong>in</strong>dicators will therefore never be comprehensive. Rema<strong>in</strong> vigilant for other sources <strong>of</strong> conflict<br />
that are not captured by your <strong>in</strong>dicators.<br />
At the output and outcome level, <strong>in</strong>dicators should be def<strong>in</strong>ed that are relevant to the conflict and the<br />
programme. Examples are given <strong>in</strong> the table below.<br />
Table 3:<br />
Def<strong>in</strong><strong>in</strong>g Indicators to Test Conflict-Relevant Risks<br />
Box from the<br />
Results Cha<strong>in</strong><br />
Jobs created<br />
Risk<br />
Project resources are seen to (or<br />
actually do) favour some groups over<br />
others<br />
Conflict-Sensitive PSD Indicator<br />
Beneficiaries’ perceptions <strong>of</strong> fairness <strong>of</strong> job<br />
distribution.<br />
Percentage <strong>of</strong> <strong>in</strong>dividuals <strong>in</strong> project area who<br />
a) Are aware <strong>of</strong> the beneficiary target<strong>in</strong>g<br />
criteria.<br />
b) Believe that the target<strong>in</strong>g criteria are fair.<br />
37 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack,<br />
2004.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
22
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
c) Believe that the target<strong>in</strong>g criteria were<br />
correctly applied.<br />
SME revenues<br />
<strong>in</strong>crease<br />
Bus<strong>in</strong>ess<br />
membership<br />
organizations<br />
(BMOs)<br />
attract new<br />
members<br />
Significant imbalance <strong>in</strong> <strong>in</strong>come<br />
change between parties to the conflict<br />
Parties to the conflict are excluded<br />
from the BMO<br />
BMO members compete for project<br />
resources (e.g. tra<strong>in</strong><strong>in</strong>g, per diems).<br />
Change <strong>in</strong> SME net <strong>in</strong>come (disaggregated by<br />
relevant party to the conflict / geographic<br />
region)<br />
Participation <strong>in</strong> BMO (disaggregated by<br />
relevant party to the conflict / geographic<br />
region)<br />
Percentage change <strong>in</strong> will<strong>in</strong>gness to<br />
economically <strong>in</strong>teract with members <strong>of</strong><br />
conflict<strong>in</strong>g communities<br />
Reasons cited by non-members <strong>of</strong> BMOs for<br />
not jo<strong>in</strong><strong>in</strong>g.<br />
Number <strong>of</strong> conflicts caused by competition<br />
for project resources.<br />
2.1.3 <strong>Measur<strong>in</strong>g</strong> positive effects <strong>of</strong> the project on the conflict<br />
PSD projects wish<strong>in</strong>g to have a positive impact on conflict should make this explicit <strong>in</strong> their results cha<strong>in</strong>s<br />
and develop related <strong>in</strong>dicators that are <strong>in</strong>formed by the results <strong>of</strong> a conflict assessment and stakeholder<br />
feedback. This ensures that your conflict <strong>in</strong>dicators are closely tied to the logic <strong>of</strong> your project. It helps<br />
to avoid vague, overarch<strong>in</strong>g <strong>in</strong>dicators, while allow<strong>in</strong>g you to trace changes related to your programme<br />
and build a credible story for why you believe you have had a positive impact.<br />
The table below illustrates <strong>in</strong>dicators for three simplified results cha<strong>in</strong>s. 38 Each one depicts a different<br />
strategy for <strong>in</strong>creas<strong>in</strong>g peace and stability based on vary<strong>in</strong>g assumptions about the source <strong>of</strong> conflict.<br />
Different <strong>in</strong>dicators are therefore required to assess the results cha<strong>in</strong>s’ validity. A range <strong>of</strong> <strong>in</strong>dicators are<br />
listed below for the results <strong>in</strong> the black and white boxes.<br />
38 In practice, results cha<strong>in</strong>s are likely to have multiple levels and branches, as does this one.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
23
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Table 4: Sample Indicators for <strong>Measur<strong>in</strong>g</strong> Positive Effects on Conflict 39<br />
Results Cha<strong>in</strong> Result 1 (Middle black box) Result 2 (Middle white box)<br />
Increase <strong>in</strong> peace and<br />
# <strong>of</strong> bus<strong>in</strong>esses jo<strong>in</strong>tly<br />
stability<br />
owned by members <strong>of</strong><br />
conflict<strong>in</strong>g communities<br />
% change <strong>in</strong><br />
Increase <strong>in</strong> trust between<br />
members <strong>of</strong> conflict<strong>in</strong>g<br />
groups<br />
Increase <strong>in</strong> economic<br />
<strong>in</strong>teractions between<br />
members <strong>of</strong> conflict<strong>in</strong>g<br />
groups<br />
Project supports bus<strong>in</strong>ess<br />
partnership and l<strong>in</strong>kage<br />
forums<br />
Increase <strong>in</strong> peace and<br />
stability<br />
Members <strong>of</strong> conflict<strong>in</strong>g<br />
group have economic<br />
<strong>in</strong>terest <strong>in</strong> peace<br />
Increase <strong>in</strong> economic<br />
<strong>in</strong>teractions between<br />
members <strong>of</strong> conflict<strong>in</strong>g<br />
groups<br />
Project supports bus<strong>in</strong>ess<br />
partnership and l<strong>in</strong>kage<br />
forums<br />
level/frequency/type <strong>of</strong><br />
economic <strong>in</strong>teraction<br />
between members <strong>of</strong><br />
conflict<strong>in</strong>g communities<br />
# <strong>of</strong> bus<strong>in</strong>esses jo<strong>in</strong>tly<br />
owned by members <strong>of</strong><br />
conflict<strong>in</strong>g communities<br />
% change <strong>in</strong><br />
level/frequency/type <strong>of</strong><br />
economic <strong>in</strong>teraction<br />
between members <strong>of</strong><br />
conflict<strong>in</strong>g communities<br />
% change <strong>in</strong> negative<br />
stereotypes about members<br />
<strong>of</strong> “oppos<strong>in</strong>g” groups<br />
% change <strong>in</strong> levels <strong>of</strong> trust<br />
between members <strong>of</strong><br />
different groups<br />
% <strong>of</strong> ex-combatants who, at<br />
the end <strong>of</strong> year X, rout<strong>in</strong>ely<br />
identify themselves as<br />
members <strong>of</strong> the larger<br />
community rather than<br />
belong<strong>in</strong>g to one group or<br />
faction<br />
% change <strong>in</strong> perceived cost <strong>of</strong><br />
conflict with members <strong>of</strong><br />
community X<br />
% change <strong>in</strong> the number <strong>of</strong><br />
people who can articulate a<br />
concrete, tangible benefit<br />
from <strong>in</strong>teract<strong>in</strong>g with an<br />
“oppos<strong>in</strong>g” group.<br />
% change <strong>in</strong> perceived<br />
benefits from economic<br />
<strong>in</strong>teraction with members <strong>of</strong><br />
community X<br />
% change <strong>in</strong> # <strong>of</strong> people who<br />
believe that their economic<br />
well-be<strong>in</strong>g and/or future is<br />
dependent on their economic<br />
relationship with the other<br />
group<br />
39 Drawn from Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict &<br />
Economics: Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact, 2011; DFID, <strong>Measur<strong>in</strong>g</strong> and manag<strong>in</strong>g for results <strong>in</strong> fragile and<br />
conflicted affected states, 2012; and <strong>in</strong>terviewees.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
24
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Increase <strong>in</strong> peace and<br />
stability<br />
Participants have more<br />
economic <strong>in</strong>centives to<br />
avoid war<br />
Participants have more jobs,<br />
<strong>in</strong>creased <strong>in</strong>come<br />
Project supports growth <strong>in</strong><br />
dynamic market systems<br />
% change <strong>in</strong> <strong>in</strong>come<br />
# <strong>of</strong> jobs created<br />
% change <strong>in</strong> satisfaction with<br />
livelihoods<br />
% <strong>of</strong> demobilised excombatants<br />
ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>g an<br />
<strong>in</strong>dependent livelihood after<br />
X year(s)<br />
% change <strong>in</strong> perceived cost <strong>of</strong><br />
conflict<br />
% change <strong>in</strong> # <strong>of</strong> people who<br />
believe that they will <strong>in</strong>cur<br />
economic losses if violence<br />
breaks out<br />
% change <strong>in</strong> # <strong>of</strong> people who<br />
see violence as a way to earn<br />
money<br />
% change <strong>in</strong> # <strong>of</strong> people who<br />
cite economic reasons for not<br />
support<strong>in</strong>g violence<br />
The top level <strong>in</strong> each <strong>of</strong> the above results cha<strong>in</strong>s is ‘<strong>in</strong>crease <strong>in</strong> peace and stability’. When there are<br />
many factors <strong>in</strong>fluenc<strong>in</strong>g this results, an <strong>in</strong>dex <strong>in</strong>dicator can aggregate changes across multiple drivers <strong>of</strong><br />
conflict. The appropriate <strong>in</strong>dicators to <strong>in</strong>clude will vary based on the context, but could <strong>in</strong>clude:<br />
% reduction <strong>in</strong> perceptions that conflict is likely<br />
% <strong>of</strong> the population who perceive improv<strong>in</strong>g security situation, disaggregated as appropriate<br />
# <strong>of</strong> violent confrontations and extrajudicial kill<strong>in</strong>gs over land, water, or graz<strong>in</strong>g rights (by identity<br />
group)<br />
% change <strong>in</strong> attitudes toward peace and violence<br />
% change <strong>in</strong> perception <strong>of</strong> personal safety<br />
2.2 <strong>Measur<strong>in</strong>g</strong> the universal impact <strong>in</strong>dicators (Control Po<strong>in</strong>t 2.2)<br />
The <strong>in</strong>clusion <strong>of</strong> the three universal impact <strong>in</strong>dicators – scale, <strong>in</strong>come, employment – <strong>in</strong> monitor<strong>in</strong>g<br />
systems supports the aggregation <strong>of</strong> PSD impacts across multiple projects. Their application <strong>in</strong> CAEs is<br />
important, as there is evidence that generat<strong>in</strong>g <strong>in</strong>come and employment can contribute to <strong>in</strong>creased<br />
stability. 40 As with conflict <strong>in</strong>dicators, it is particularly critical to disaggregate employment and <strong>in</strong>come<br />
results to assess the distribution <strong>of</strong> benefits.<br />
There is general consensus the uniqueness <strong>of</strong> each CAE prevents the development <strong>of</strong> universal<br />
<strong>in</strong>dicators for peacebuild<strong>in</strong>g. 41 Rather, other impact-level <strong>in</strong>dicators should be developed as needed<br />
based on the specific nature <strong>of</strong> the conflict. This will be particularly important for PSD projects with a<br />
peacebuild<strong>in</strong>g objective.<br />
40 ECOPA, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict-affected countries, Phase 2: Case studies and<br />
recommendations, August 2011, 24.<br />
41 ECOPA, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict-affected countries, Phase 2: Case studies and<br />
recommendations, August 2011, 106.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
25
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
2.3 Assess<strong>in</strong>g likelihood <strong>of</strong> susta<strong>in</strong>able impact (Control Po<strong>in</strong>t 2.3)<br />
The <strong>in</strong>clusion <strong>of</strong> <strong>in</strong>dicators that assess the susta<strong>in</strong>ability <strong>of</strong> project <strong>in</strong>terventions pushes projects to<br />
assess whether development <strong>in</strong>terventions will lead to last<strong>in</strong>g change. The unpredictability and<br />
fluctuations <strong>in</strong> CAEs makes the use <strong>of</strong> <strong>in</strong>dicators <strong>of</strong> susta<strong>in</strong>ability particularly important. Given the<br />
devastat<strong>in</strong>g impact that conflict can have upon the private sector, monitor<strong>in</strong>g the potential for conflict is<br />
a critical aspect <strong>of</strong> susta<strong>in</strong>ability. The <strong>in</strong>dicators presented above can help <strong>in</strong> accomplish<strong>in</strong>g this.<br />
2.4 Manag<strong>in</strong>g <strong>in</strong>dicators (Control Po<strong>in</strong>t 2.4)<br />
Dur<strong>in</strong>g the course <strong>of</strong> project implementation, changes <strong>in</strong><br />
the CAE context and project learn<strong>in</strong>g based on<br />
implementation results may both require changes to the<br />
results cha<strong>in</strong>s. This may require different or additional<br />
project <strong>in</strong>dicators.<br />
Relevant to<br />
project logic<br />
A common mistake <strong>in</strong> CAEs (and elsewhere) is to<br />
overburden staff with <strong>in</strong>dicators. Ensure <strong>in</strong>dicators are<br />
relevant to the logic <strong>of</strong> your project and that there is<br />
capacity and budget to collect, analyse and use them.<br />
Hard<br />
collect<br />
to<br />
Easy<br />
collect<br />
to<br />
One way to manage this is to plot all current <strong>in</strong>dicators on<br />
a grid like the one to the right, potentially as part <strong>of</strong> a<br />
results measurement workshop. 42 Select <strong>in</strong>dicators <strong>in</strong> the<br />
top right <strong>of</strong> the graph that are relevant to the project<br />
logic and easy to collect while avoid<strong>in</strong>g those <strong>in</strong> the<br />
bottom left.<br />
Irrelevant to<br />
project logic<br />
Preference<br />
Figure 4: Indicator Selection Tool<br />
Tip!<br />
Community members are <strong>of</strong>ten are best placed to understand what would represent an improvement <strong>in</strong><br />
peace <strong>in</strong> their particular context. Runn<strong>in</strong>g workshops with them can be an important source <strong>of</strong><br />
<strong>in</strong>formation <strong>in</strong> the <strong>in</strong>dicator development process. 43 For <strong>in</strong>stance, <strong>in</strong> Nepal a community peacebuild<strong>in</strong>g<br />
project consulted community members to develop <strong>in</strong>dicators for an economic development project.<br />
They learned that be<strong>in</strong>g viewed as credit-worthy by local moneylenders was a critical early <strong>in</strong>dicator that<br />
the ultra poor had made economic ga<strong>in</strong>s. 44<br />
2.5 Projections (Control Po<strong>in</strong>ts 2.5)<br />
Predict<strong>in</strong>g the results <strong>of</strong> programme <strong>in</strong>terventions provides staff with targets and a way to assess<br />
whether <strong>in</strong>terventions are proceed<strong>in</strong>g as expected. They also enable up-front assessments <strong>of</strong> whether<br />
the expected results <strong>of</strong> specific <strong>in</strong>terventions are worth their <strong>in</strong>vestment.<br />
42 RAND Europe, <strong>in</strong>terview September 25, 2012.<br />
43 Sartorius, Rolf and Christopher Carver. Monitor<strong>in</strong>g, Evaluation and Learn<strong>in</strong>g for Fragile States and Peacebuild<strong>in</strong>g<br />
Programs: Practical Tools for Improv<strong>in</strong>g Program Performance and Results, undated, 44.<br />
44 McGrew, Laura, Community Peace build<strong>in</strong>g <strong>in</strong> Nepal: Lessons Learned, 2006, 11.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
26
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Sett<strong>in</strong>g projections <strong>in</strong> a CAE requires mak<strong>in</strong>g adjustments for the difficult operat<strong>in</strong>g environment.<br />
Projects frequently take longer to achieve their goals <strong>in</strong> CAEs. For example, a survey <strong>of</strong> International<br />
F<strong>in</strong>ance Corporation (IFC) projects found that just 17% achieved their timeframe targets <strong>in</strong> environments<br />
where conflict was <strong>in</strong>creas<strong>in</strong>g, compared to approximately 50% <strong>of</strong> projects <strong>in</strong> non-CAEs. 45 To<br />
compensate, projects are advised to set realistic targets. The most common adjustments are to reduce<br />
expected deliverables and/or lengthen the timeframe for achievements. The many delays, disruptions<br />
and setbacks that frequently occur <strong>in</strong> CAEs can be reflected by sett<strong>in</strong>g longer timeframes for<br />
deliverables. Another strategy is to reduce the duration <strong>of</strong> time over which projections are made so as<br />
to account for the fast-chang<strong>in</strong>g environment.<br />
2.6 Key Resources<br />
Agoglia et al. <strong>Measur<strong>in</strong>g</strong> Progress <strong>in</strong> Conflict Environments. Provides a significant volume <strong>of</strong> outcome<br />
<strong>in</strong>dicators around five aspects <strong>of</strong> a conflict, <strong>in</strong>clud<strong>in</strong>g the existence <strong>of</strong> a susta<strong>in</strong>able economy. These are<br />
high-level <strong>in</strong>dicators for gaug<strong>in</strong>g the overall stability <strong>of</strong> a situation, rather than the direct success <strong>of</strong> an<br />
<strong>in</strong>dividual project<br />
CDA. Indications for Assess<strong>in</strong>g Aid’s Impact on Conflict. Offers <strong>in</strong>dicators to measure a project’s<br />
positive and negative effects on conflict.<br />
Church, Cheyanne and Mark M. Roberts. Design<strong>in</strong>g for Results: Integrat<strong>in</strong>g M&E <strong>in</strong> Conflict<br />
Transformation Programmes. Includes general guidel<strong>in</strong>es on monitor<strong>in</strong>g and evaluat<strong>in</strong>g <strong>in</strong> CAEs.<br />
Chapter Four <strong>in</strong>cludes a good discussion on <strong>in</strong>dicators and some helpful examples.<br />
DFID. Interim Guidance Note: <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict-Affected<br />
States and Situations. Lists <strong>in</strong>dicators for high level objectives and possible data sources <strong>in</strong> its Annex A.<br />
More focused on country programmes than projects.<br />
Mercy Corps. Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict & Economics:<br />
Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Provides a menu <strong>of</strong> 29 peace and conflict <strong>in</strong>dicators and 49<br />
economics and conflict <strong>in</strong>dicators. Each <strong>in</strong>dicator is def<strong>in</strong>ed and possible ways to disaggregate the data<br />
are suggested. There is a useful discussion <strong>of</strong> how the <strong>in</strong>dicators were selected and measured.<br />
3 <strong>Measur<strong>in</strong>g</strong> Changes <strong>in</strong> Indicators<br />
It is critical to measure changes <strong>in</strong> <strong>in</strong>dicators to assess project performance and impact. However, there<br />
are many challenges to data collection <strong>in</strong> CAEs. These challenges are frequently present <strong>in</strong> humanitarian<br />
crises and develop<strong>in</strong>g countries more generally, and so may be relevant for practitioners implement<strong>in</strong>g<br />
the <strong>DCED</strong> Standard <strong>in</strong> those contexts. Challenges <strong>in</strong>clude: 46<br />
45 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />
Phase 1: Formative research and key issue diagnosis, undated, 6.<br />
46 For further reference, see DFID, Manag<strong>in</strong>g Results <strong>in</strong> Conflict Affected States, Annex J.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
27
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Limited access. Risks <strong>of</strong> harm to project staff and project clients may limit the ability to access<br />
project sites and meet with project clients. Political <strong>in</strong>stitutions may also not allow access, or<br />
restrict the questions that can be asked.<br />
Weak capacity. It is <strong>of</strong>ten challeng<strong>in</strong>g to f<strong>in</strong>d and reta<strong>in</strong> effective results measurement staff, 47<br />
while low capacity among third parties (universities, survey firms, consultants) may reduce the<br />
ability to outsource the collection <strong>of</strong> data.<br />
Limited secondary data. Government <strong>in</strong>stitutions may not collect or publish accurate data<br />
ow<strong>in</strong>g to limited capacity and resources. In particular, there may be limited data on populations,<br />
which reduces the validity <strong>of</strong> sampl<strong>in</strong>g techniques. This may necessitate an <strong>in</strong>creased amount <strong>of</strong><br />
primary data collection.<br />
Greater monitor<strong>in</strong>g costs. A lack <strong>of</strong> <strong>in</strong>frastructure (e.g. roads, telecommunications networks),<br />
security costs and recruitment challenges will typically require higher spend<strong>in</strong>g on results<br />
measurement. 48<br />
Sensitivity <strong>of</strong> data. Clients and partners may be put at risk if project data is accessed by parties<br />
to the conflict. Obta<strong>in</strong><strong>in</strong>g sensitive <strong>in</strong>formation (e.g. lists <strong>of</strong> beneficiaries, their locations,<br />
household <strong>in</strong>come, assets levels, ethnicity) can expose <strong>in</strong>dividuals to danger or create jealousy<br />
among those who did not benefit. This may also cause unwill<strong>in</strong>gness among beneficiaries and<br />
other stakeholders to share <strong>in</strong>formation. Participants <strong>in</strong> focus group discussions held by Mercy<br />
Corps <strong>in</strong> Indonesia, for <strong>in</strong>stance, requested that it be known that they had not benefited from<br />
their participation <strong>in</strong> the discussion. 49<br />
Data <strong>in</strong>terference. Local staff or partners may be biased <strong>in</strong> their report<strong>in</strong>g. For example, <strong>in</strong> an<br />
area with strong clan loyalties they may feel under pressure to portray their own ethnic group<br />
positively, or to report results that will cause a project to cont<strong>in</strong>ue operat<strong>in</strong>g <strong>in</strong> their home area.<br />
Slower implementation and results. It takes longer to achieve and measure results relative to<br />
other contexts. 50 For example, the Department for International <strong>Development</strong> (DFID) notes that<br />
it can require 15 years to build peace <strong>in</strong> the best <strong>of</strong> situations, which is far longer than the<br />
typical project report<strong>in</strong>g cycle. 51<br />
3.1 Collect<strong>in</strong>g basel<strong>in</strong>e <strong>in</strong>formation (Control Po<strong>in</strong>t 3.1)<br />
Basel<strong>in</strong>es are an essential tool for results measurement. They capture the situation at the beg<strong>in</strong>n<strong>in</strong>g <strong>of</strong><br />
an <strong>in</strong>tervention, enabl<strong>in</strong>g change <strong>in</strong> key <strong>in</strong>dicators to be detected. They can also capture additional<br />
<strong>in</strong>formation about the context, such as further explor<strong>in</strong>g issues that arose dur<strong>in</strong>g conflict analysis.<br />
47 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />
Phase 1: Formative research and key issue diagnosis, undated, 21.<br />
48 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />
Phase 1: Formative research and key issue diagnosis, undated, 20-21.<br />
49 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions: Indonesia Case Study Report,<br />
2011, 9.<br />
50 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />
Phase 1: Formative research and key issue diagnosis, undated, 20.<br />
51 DFID, <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict-Affected States and Situations: How To Note,<br />
2012, 3.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
28
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
However, basel<strong>in</strong>es <strong>in</strong> CAEs can be particularly complex due to the challenges listed above. Moreover,<br />
projects <strong>in</strong> CAEs are more likely to require a flexible implementation strategy that <strong>in</strong>volves chang<strong>in</strong>g<br />
their activities, outputs and outcomes as the project goes on. This could conceivably render orig<strong>in</strong>al<br />
basel<strong>in</strong>e <strong>in</strong>formation irrelevant. The <strong>DCED</strong> Standard emphasizes the importance <strong>of</strong> articulat<strong>in</strong>g specific<br />
results cha<strong>in</strong>s and <strong>in</strong>dicators for each project <strong>in</strong>tervention. Consequently, <strong>in</strong>stead <strong>of</strong> conduct<strong>in</strong>g a s<strong>in</strong>gle,<br />
large project-wide basel<strong>in</strong>e, each <strong>in</strong>tervention may have its own smaller basel<strong>in</strong>e. This enables projects<br />
to avoid discard<strong>in</strong>g all basel<strong>in</strong>e data if, for <strong>in</strong>stance, violent upheaval renders some <strong>of</strong> a project’s<br />
<strong>in</strong>terventions unfeasible.<br />
Retrospective basel<strong>in</strong>es are basel<strong>in</strong>es conducted after the <strong>in</strong>tervention has already started, by ask<strong>in</strong>g<br />
participants to recall the situation before the start <strong>of</strong> the project. They may be the only option to collect<br />
<strong>in</strong>formation about the past, although their use is not ideal and can produce biased and erroneous<br />
results. 52 If us<strong>in</strong>g retrospective basel<strong>in</strong>es, it is important to carefully consider both the time lag s<strong>in</strong>ce<br />
the event <strong>in</strong> question has occurred and the type <strong>of</strong> event that you are ask<strong>in</strong>g about. The type <strong>of</strong><br />
experience determ<strong>in</strong>es how well it can be recalled. For example, questions about actions performed<br />
directly by the respondent are more likely to be recalled. Information that was heard or read – for<br />
example, market access <strong>in</strong>formation provided by a service provider <strong>in</strong> a phone call – will be forgotten<br />
more quickly. A rough guide is to not ask recall questions about events that occurred over a year ago, as<br />
memory becomes <strong>in</strong>creas<strong>in</strong>gly unreliable.<br />
3.2 Good research practices (Control Po<strong>in</strong>t 3.2)<br />
While the standard good research practices that are listed <strong>in</strong> the <strong>DCED</strong>’s Guide to Conduct<strong>in</strong>g Research 53<br />
apply to CAEs, there are other important considerations. The challenges <strong>of</strong> work<strong>in</strong>g <strong>in</strong> CAEs mean that<br />
the acceptable standard for research practices, <strong>in</strong>clud<strong>in</strong>g sample sizes and attribution measurement,<br />
(see the section on attribution, below) may be less rigorous. Less exact<strong>in</strong>g standards <strong>of</strong> practice are<br />
sometimes all that is possible.<br />
This section presents good practices for design<strong>in</strong>g research tools and collection <strong>in</strong>formation <strong>in</strong> CAEs.<br />
Ultimately, ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>g a focus on the three pr<strong>in</strong>ciples for success (simplicity, flexibility, sensitivity) is<br />
key.<br />
Some tips for select<strong>in</strong>g questions and <strong>in</strong>formation tools are:<br />
Use short, simple questions. Simple questions are more easily understood by staff and users<br />
and therefore provide better results. Overly complicated questions can greatly <strong>in</strong>crease the<br />
workload both <strong>in</strong> data collection and analysis. Similarly, close-ended survey questions are <strong>of</strong>ten<br />
preferable for their greater ease <strong>of</strong> cod<strong>in</strong>g and analys<strong>in</strong>g. Focus group discussions and surveys<br />
also should be <strong>of</strong> a manageable size. One practitioner suggests that 40 survey questions and 10<br />
focus group discussion questions should be the maximum <strong>in</strong> order to not overwhelm data<br />
52 Ravaillon, Mart<strong>in</strong>, Can We Trust Shoestr<strong>in</strong>g Evaluations Policy Research Work<strong>in</strong>g Paper 5983, 2012.<br />
53 Donor Committee for Enterprise <strong>Development</strong>, <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong><br />
Implementation Guidel<strong>in</strong>es, Version 1g, 5 th March 2010, 26-27.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
29
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
collection teams and respondents. 54 Shorter surveys also typically allow a greater number <strong>of</strong><br />
<strong>in</strong>terviews to be conducted and thus <strong>in</strong>crease the total potential sample size.<br />
Complement the measurement <strong>of</strong> beliefs and attitudes when possible. Ask<strong>in</strong>g about beliefs<br />
and attitudes can provide helpful <strong>in</strong>formation about changes that are otherwise difficult to<br />
measure. However beliefs and attitudes can be <strong>in</strong>fluenced by many factors, can change based<br />
on events occurr<strong>in</strong>g around the time <strong>of</strong> the survey and may be tailored to fit the expected biases<br />
<strong>of</strong> the <strong>in</strong>terviewers. Ask<strong>in</strong>g about behaviour can help to complement this <strong>in</strong>formation. For<br />
example, ask<strong>in</strong>g about levels <strong>of</strong> trust between groups directly may not elicit useful responses.<br />
Behavioural questions such as “<strong>in</strong> the last year, have you bought anyth<strong>in</strong>g from a member <strong>of</strong><br />
group X” can be more reliable.<br />
Use proxy <strong>in</strong>dicators as necessary to be conflict sensitive. In some cases, it might be dangerous<br />
or <strong>in</strong>appropriate to directly ask about conflict related factors. In this case, you can use proxy<br />
<strong>in</strong>dicators. For <strong>in</strong>stance, the breadth <strong>of</strong> <strong>in</strong>ter-group economic transactions, rate <strong>of</strong><br />
<strong>in</strong>termarriage or collaboration on decision-mak<strong>in</strong>g bodies are all potential proxy <strong>in</strong>dicators to<br />
measure the state <strong>of</strong> <strong>in</strong>ter-group acceptance. 55<br />
Field test tools. It is important to test tools with staff, partners and community members before<br />
widespread roll out. Many methods look easy on paper but require f<strong>in</strong>e tun<strong>in</strong>g once you start to<br />
use them. Field-test<strong>in</strong>g tools will help ensure that questions are appropriate, conflict-sensitive,<br />
and simple to adm<strong>in</strong>ister.<br />
Some tips for data-collection are:<br />
Build trust. Results measurement frequently relies on discussions or surveys with those affected<br />
by the <strong>in</strong>tervention, such as partners and the communities who utilise products or services. If<br />
these groups do not trust the project staff, they may refuse to answer questions or choose not<br />
to answer honestly. This is especially relevant when <strong>in</strong>vestigat<strong>in</strong>g sensitive topics related to<br />
conflict. Ask<strong>in</strong>g for direct feedback on project impact may also be challeng<strong>in</strong>g, as this will be<br />
perceived to affect future fund<strong>in</strong>g <strong>in</strong> the area. Consequently, programmes should build strong<br />
relationships with partners and the community.<br />
Provide tra<strong>in</strong><strong>in</strong>g <strong>in</strong> conflict-sensitive monitor<strong>in</strong>g approaches. To mitigate the risk <strong>of</strong> <strong>in</strong>flam<strong>in</strong>g<br />
sensitivities through data collection, adequately tra<strong>in</strong> the <strong>in</strong>dividuals collect<strong>in</strong>g monitor<strong>in</strong>g<br />
<strong>in</strong>formation. Tra<strong>in</strong><strong>in</strong>g should emphasize provid<strong>in</strong>g the option for participants to not answer<br />
questions if they wish, keep<strong>in</strong>g answers confidential, and tak<strong>in</strong>g care when ask<strong>in</strong>g questions<br />
which may be sensitive. In certa<strong>in</strong> contexts the use <strong>of</strong> terms like conflict or clan are highly<br />
<strong>in</strong>flammatory, while <strong>in</strong> others – such as was the experience <strong>of</strong> Mercy Corps <strong>in</strong> Uganda 56 – they<br />
are not.<br />
54 Interview with Jenny Vaughan (Mercy Corps), October 1, 2012.<br />
55 DFID, Work<strong>in</strong>g Effectively <strong>in</strong> Conflict-affected and Fragile Situations: Brief<strong>in</strong>g Paper I: Monitor<strong>in</strong>g and Evaluation,<br />
9.<br />
56 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions: Uganda Case Study Report,<br />
2011, 9.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
30
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Use mobile technologies. Technology can reduce or elim<strong>in</strong>ate the need for physical visits to<br />
collect <strong>in</strong>formation, thereby reduc<strong>in</strong>g both cost and risk. Mobile phones are common and cheap<br />
<strong>in</strong> most CAEs, and <strong>of</strong>ten provide a good technological solution to <strong>in</strong>formation gather<strong>in</strong>g. 57<br />
Case Study: Collect<strong>in</strong>g Conflict-Related Data through SMS Report<strong>in</strong>g 58<br />
Columbia University’s Voix-des-Kivus project piloted a data-gather<strong>in</strong>g system <strong>in</strong> Eastern Congo <strong>in</strong> which<br />
reporters <strong>in</strong> randomly selected villages reported on events <strong>in</strong> real time. They selected three reporters <strong>in</strong><br />
each <strong>of</strong> 18 villages; one traditional leader, one women’s representative, and one elected by the<br />
community. Each received $1.5 a week <strong>in</strong> return for send<strong>in</strong>g at least one text report<strong>in</strong>g on current<br />
events. A pre-agreed cod<strong>in</strong>g system allowed for easy and systemised data collection. For example, a text<br />
beg<strong>in</strong>n<strong>in</strong>g ‘46’ <strong>in</strong>dicated ethnic violence.<br />
In total, 4,623 unique events were reported – many were reported by more than one person. 30% <strong>of</strong><br />
these were conflict-related. The researchers found that this method was much cheaper and more<br />
reliable than traditional questionnaire data. It works <strong>in</strong> places where survey teams cannot reach, gathers<br />
<strong>in</strong>formation <strong>in</strong> real time, and builds a relationship <strong>of</strong> trust between the project and the reporters,<br />
enabl<strong>in</strong>g higher-quality <strong>in</strong>formation to be collected. By comb<strong>in</strong><strong>in</strong>g this rich dataset with a randomised<br />
<strong>in</strong>tervention run by the IRC, Columbia University found aid projects reduced conflict. There is the<br />
potential for a PSD project to imitate aspects <strong>of</strong> this pilot. It improves accountability to the local<br />
community, allow<strong>in</strong>g them to text suggestions and compla<strong>in</strong>ts. Moreover, it can be <strong>in</strong>tegrated <strong>in</strong>to the<br />
results measurement system to monitor the effects <strong>of</strong> the project on conflict <strong>in</strong> real time.<br />
Build flexibility <strong>in</strong>to the monitor<strong>in</strong>g schedule. Schedule additional time for data collection to<br />
allow for potential disruptions. Develop an alternate survey plan with a second sample already<br />
selected <strong>in</strong> case security issues prevent the <strong>in</strong>itial plan from proceed<strong>in</strong>g.<br />
Select a conflict-appropriate sample. Depend<strong>in</strong>g upon the type <strong>of</strong> <strong>in</strong>formation you wish to<br />
collect and conflict-related sensitivities, a purposive sample may be preferable to a random<br />
sample. A purposive sample can allow you to capture data from perspectives that are important<br />
to the conflict but that may be missed by us<strong>in</strong>g a randomized approach.<br />
Build a balanced monitor<strong>in</strong>g team. Us<strong>in</strong>g a mix <strong>of</strong> local and non-local staff br<strong>in</strong>gs both an<br />
<strong>in</strong>dependent perspective and local knowledge to monitor<strong>in</strong>g. This can br<strong>in</strong>g <strong>in</strong> external<br />
expertise and reduce bias <strong>in</strong> data collection while benefit<strong>in</strong>g from local knowledge and<br />
experience. It can also build the capacity <strong>of</strong> the local staff if well managed.<br />
Protect aga<strong>in</strong>st loss <strong>of</strong> your sample. In contexts where migration is high and where there is<br />
<strong>in</strong>security, projects may be unable to re-survey some members <strong>of</strong> their basel<strong>in</strong>e. Ways to<br />
protect aga<strong>in</strong>st the loss <strong>of</strong> one’s sample <strong>in</strong>clud<strong>in</strong>g oversampl<strong>in</strong>g (i.e. survey<strong>in</strong>g a larger<br />
population than is statistically required) to ga<strong>in</strong> a larger <strong>in</strong>itial sample and request<strong>in</strong>g a contact<br />
57 One popular free s<strong>of</strong>tware for conduct<strong>in</strong>g polls and surveys through mobile phone is Frontl<strong>in</strong>e SMS.<br />
58 W<strong>in</strong>dt and Humphreys, Crowdseed<strong>in</strong>g Conflict Data: An Application <strong>of</strong> an SMS-based Data System to Estimate<br />
the Conflict Effects <strong>of</strong> <strong>Development</strong> Aid<br />
W<strong>in</strong>dt and Humphreys, Voix des Kivus: Reflections on a Crowdseed<strong>in</strong>g Approach to Conflict Event Data Gather<strong>in</strong>g,<br />
2012.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
31
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
person who would know where participants have gone if they relocate. The IRC’s large sample<br />
size across multiple regions <strong>in</strong> the Democratic Republic <strong>of</strong> the Congo allowed a randomized<br />
control trial to cont<strong>in</strong>ue despite political issues <strong>in</strong> one region that ended its data collection<br />
there. 59 The IRC has also found that the basel<strong>in</strong>e data collected on <strong>in</strong>dividuals who leave can<br />
still serve the purpose <strong>of</strong> compar<strong>in</strong>g the characteristics <strong>of</strong> more mobile populations versus those<br />
who stayed <strong>in</strong> the program.<br />
Triangulate <strong>in</strong>formation. Given the frequent lack <strong>of</strong> high quality data <strong>in</strong> a CAE, all <strong>in</strong>formation<br />
should be triangulated where feasible. This means that multiple methods should be used to<br />
improve the reliability <strong>of</strong> the results. Often qualitative, quantitative and participatory methods<br />
can be used together to triangulate <strong>in</strong>formation, as can <strong>in</strong>terview<strong>in</strong>g multiple sources.<br />
Triangulat<strong>in</strong>g <strong>in</strong>formation <strong>in</strong> the SEED programme<br />
The SEED programme <strong>in</strong> Somalia, implemented by FAO, uses a variety <strong>of</strong> ways to triangulate<br />
<strong>in</strong>formation, <strong>in</strong>clud<strong>in</strong>g:<br />
Partner reports. Implement<strong>in</strong>g NGOs are required to submit regular reports on their progress.<br />
Random calls. A dedicated unit with<strong>in</strong> FAO calls a random sample <strong>of</strong> between 2-3% <strong>of</strong> all<br />
beneficiaries, to verify that reported results have been achieved. They also call key <strong>in</strong>formants,<br />
such as elders or market venders.<br />
Field monitors. The monitor<strong>in</strong>g unit has 25 local field monitors, who can access <strong>in</strong>secure areas<br />
and verify that the activities are tak<strong>in</strong>g place<br />
Satellite imagery. FAO have a large programme to rehabilitate irrigation canals. They require<br />
partners to map out the planned canal route with a GPS device, and FAO use satellite imagery, to<br />
assess the progress <strong>of</strong> the project.<br />
Biometrics. FAO are trial<strong>in</strong>g biometric identification track<strong>in</strong>g via f<strong>in</strong>gerpr<strong>in</strong>t scanners for cash-forwork<br />
programme recipients.<br />
More <strong>in</strong>formation is available <strong>in</strong> the SEED case study, available on the <strong>DCED</strong> website.<br />
Outsource data collection. Outsourc<strong>in</strong>g data collection to local entities can allow implement<strong>in</strong>g<br />
agencies to collect data <strong>in</strong> areas where implement<strong>in</strong>g agency staff lack access. It can also reduce<br />
the political pressure on project implementers to report positive results by creat<strong>in</strong>g a formal<br />
barrier <strong>in</strong> data collection. Effective outsourc<strong>in</strong>g does however require effective oversight by the<br />
implement<strong>in</strong>g agency.<br />
3.3 Qualitative <strong>in</strong>formation (Control Po<strong>in</strong>t 3.3)<br />
Qualitative <strong>in</strong>formation is necessary <strong>in</strong> any context, but especially <strong>in</strong> CAEs, where perceptions, beliefs<br />
and op<strong>in</strong>ions are essential to the success <strong>of</strong> a project. Qualitative <strong>in</strong>formation typically requires more<br />
time to collect and analyse than qualitative <strong>in</strong>formation. It is therefore important to carefully consider<br />
when qualitative question<strong>in</strong>g is most important and use it judiciously. Too many qualitative questions<br />
on surveys, for <strong>in</strong>stance, <strong>in</strong>creases the time required for <strong>in</strong>terviews and subsequent analysis. This may<br />
59 Interview with IRC, Jeannie Annan, September 26, 2012.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
32
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
limit your sample size and therefore your ability to extrapolate from your f<strong>in</strong>d<strong>in</strong>gs to the broader<br />
population.<br />
One common approach to collect<strong>in</strong>g qualitative <strong>in</strong>formation is to use participatory techniques that take<br />
<strong>in</strong>to account the perspectives and <strong>in</strong>sights <strong>of</strong> all stakeholders, beneficiaries and partners as well as<br />
project implementers. They typically emphasise build<strong>in</strong>g the capacity <strong>of</strong> all these groups to reflect on<br />
<strong>in</strong>terventions to analyse what worked and did not. This can be used to strengthen relationships among<br />
parties to a conflict and agreement on future collaboration.<br />
3.4 Key resources<br />
Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions: Conflict & Economics:<br />
Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Provides several conflict monitor<strong>in</strong>g tools, <strong>in</strong>clud<strong>in</strong>g a violent<br />
<strong>in</strong>cident report<strong>in</strong>g form, a disputes and dispute resolution assessment, a scored community relationship<br />
mapp<strong>in</strong>g tool, and a conflict and resource mapp<strong>in</strong>g tool.<br />
Catley, Burns, Abebe and Suji, Participatory Impact Assessment: A Guide for Practitioners, 2008,<br />
Fe<strong>in</strong>ste<strong>in</strong> Centre. Outl<strong>in</strong>es an eight-step approach to conduct<strong>in</strong>g a participatory impact assessment.<br />
Frontl<strong>in</strong>e SMS. A free s<strong>of</strong>tware for creat<strong>in</strong>g and manag<strong>in</strong>g SMS activities such as mak<strong>in</strong>g<br />
announcements, conduct<strong>in</strong>g polls, and automatically reply<strong>in</strong>g to <strong>in</strong>com<strong>in</strong>g SMS. In particular, it is<br />
excellent for manag<strong>in</strong>g large mobile phone surveys and was used <strong>in</strong> the above example by Colombia<br />
University to gather data <strong>in</strong> Eastern DRC.<br />
Muaz Jalil. Practical Guidel<strong>in</strong>es for Conduct<strong>in</strong>g Research. Provides a review <strong>of</strong> good practices <strong>in</strong><br />
design<strong>in</strong>g and conduct<strong>in</strong>g research <strong>in</strong> l<strong>in</strong>e with the <strong>DCED</strong> Standard.<br />
Sartorius, Rolf and Christopher Carver. Monitor<strong>in</strong>g, Evaluation and Learn<strong>in</strong>g for Fragile States and<br />
Peacebuild<strong>in</strong>g Programs: Practical Tools for Improv<strong>in</strong>g Program Performance and Results. Offers a<br />
large number <strong>of</strong> monitor<strong>in</strong>g tools for peacebuild<strong>in</strong>g programmes, tips on measurement and sample<br />
<strong>in</strong>dicators.<br />
4 Estimat<strong>in</strong>g Attributable Changes<br />
Estimat<strong>in</strong>g attribution requires establish<strong>in</strong>g a causal l<strong>in</strong>k between an <strong>in</strong>tervention and observed changes<br />
<strong>in</strong> <strong>in</strong>dicators and account<strong>in</strong>g for what would have happened even if the program had not been operat<strong>in</strong>g<br />
(i.e. the counterfactual). This is difficult because many other factors may contribute to the change. For<br />
<strong>in</strong>stance, a 50% <strong>in</strong>crease <strong>in</strong> employment among project sites <strong>in</strong> Somalia follow<strong>in</strong>g an <strong>in</strong>tervention may<br />
be due to the project’s activities but could also be ow<strong>in</strong>g to favourable macroeconomic conditions, an<br />
<strong>in</strong>crease <strong>in</strong> illicit activities (lead<strong>in</strong>g to <strong>in</strong>creased wealth and employment), or good weather. To estimate<br />
attributable change, it is necessary to assess how much <strong>of</strong> the observed results are caused by the<br />
project and how much would have happened without it.<br />
Design<strong>in</strong>g an effective approach to attribution is essential to properly assess<strong>in</strong>g the impact <strong>of</strong> your<br />
project, as it greatly <strong>in</strong>creases the credibility <strong>of</strong> reported results. While always a challenge, <strong>in</strong> CAEs<br />
design<strong>in</strong>g an effective approach to attribution is particularly difficult because <strong>of</strong>:<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
33
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Difficulty <strong>of</strong> <strong>in</strong>formation gather<strong>in</strong>g. Prov<strong>in</strong>g attribution <strong>of</strong>ten requires gather<strong>in</strong>g additional<br />
<strong>in</strong>formation, which is particularly challeng<strong>in</strong>g <strong>in</strong> a CAE. This is particularly problematic for<br />
experimental or quasi-experimental designs that require large amounts <strong>of</strong> high quality data.<br />
Rapidly chang<strong>in</strong>g context. CAEs typically change rapidly. If conflict is ongo<strong>in</strong>g, then conditions<br />
may be rapidly deteriorat<strong>in</strong>g. If it is post-conflict, the environment may be rapidly improv<strong>in</strong>g as<br />
the country moves forward from the war. In both cases, relationships between different groups,<br />
the bus<strong>in</strong>ess environment, and external <strong>in</strong>vestment will likely be fluctuat<strong>in</strong>g greatly. This makes<br />
it harder to attribute change to your programme.<br />
More complex theories <strong>of</strong> change. The more external factors affect the outcomes you are<br />
measur<strong>in</strong>g, the harder it will be to attribute change. This is because there will be more<br />
alternative explanations for the change that you observe, and so it will be more challeng<strong>in</strong>g to<br />
show the role played by your programme. A sub-national conflict, for <strong>in</strong>stance, may be<br />
<strong>in</strong>fluenced by <strong>in</strong>ternational policies (e.g. sanctions), national policies, socio-cultural norms and<br />
many other factors. There are usually multiple causes <strong>of</strong> any change and disentangl<strong>in</strong>g the l<strong>in</strong>ks<br />
between them is extremely difficult. Whereas many PSD projects attempt simply to attribute<br />
positive economic outcomes (e.g. employment, <strong>in</strong>come) for project clients to their<br />
<strong>in</strong>terventions, those with peacebuild<strong>in</strong>g objectives <strong>of</strong>ten try to establish that those positive<br />
economic outcomes reduced or prevented conflict. The greater number <strong>of</strong> l<strong>in</strong>ks to be proven<br />
<strong>in</strong>creases the challenge <strong>of</strong> demonstrat<strong>in</strong>g attribution.<br />
The <strong>DCED</strong> Standard recognises the complexity <strong>of</strong> demonstrat<strong>in</strong>g attribution, and advocates for an<br />
approach that would conv<strong>in</strong>ce a “reasonable but sceptical observer”. 60 It recognises that there are<br />
multiple contributors to any positive (or negative) outcome, and the programme <strong>in</strong> some cases may<br />
have a relatively m<strong>in</strong>or part to play. In particular, the <strong>DCED</strong> Standard does not require ‘scientific’ pro<strong>of</strong><br />
<strong>of</strong> causality, which <strong>of</strong>ten requires randomised control trials and is generally not practical with<strong>in</strong> the<br />
constra<strong>in</strong>ts <strong>of</strong> development programmes. The <strong>DCED</strong> has published separate guidel<strong>in</strong>es on measur<strong>in</strong>g<br />
attribution that can be found here.<br />
4.1 A clear and appropriate system for attribution is <strong>in</strong> place (Control Po<strong>in</strong>t<br />
4.1)<br />
There are three key complementary approaches to estimat<strong>in</strong>g attribution <strong>in</strong> CAEs.<br />
First, use the results cha<strong>in</strong>s as the basis for attribution by validat<strong>in</strong>g each key change. This is a ‘theorybased’<br />
approach to assess<strong>in</strong>g attribution, which argues that there are several necessary conditions for<br />
claim<strong>in</strong>g attribution: 61<br />
The results cha<strong>in</strong> must be well designed, logical and evidence-based.<br />
The expected changes at every level <strong>of</strong> the results cha<strong>in</strong> must have occurred.<br />
Alternative explanations for these changes have been considered and rejected.<br />
60 Sen, Nabanita, Attribution: <strong>Measur<strong>in</strong>g</strong> attributable change caused by a programme, 2012.<br />
61 White and Phillips, Address<strong>in</strong>g attribution <strong>of</strong> cause and effect <strong>in</strong> small n impact evaluations: towards an<br />
<strong>in</strong>tegrated framework, 2012, 3ie, 14<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
34
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
If these are all met, it is reasonable to conclude that the results can be attributed to the project. The<br />
results cha<strong>in</strong> tells a ‘contribution story’ which shows how you believe you <strong>in</strong>fluenced the outcomes.<br />
Second, utilise qualitative tools for assess<strong>in</strong>g attribution. These support the use <strong>of</strong> the results cha<strong>in</strong> to<br />
demonstrate attribution, by assess<strong>in</strong>g the l<strong>in</strong>ks between the results cha<strong>in</strong>. In particular, qualitative tools<br />
are typically well suited for CAE environments <strong>in</strong> which large-scale data collection is challeng<strong>in</strong>g.<br />
Participative methods are one way to assess the difference made by the project. For example, it would<br />
be possible to ask beneficiaries to rank the different factors that contribute towards an observed<br />
outcome. The follow<strong>in</strong>g figure shows one example <strong>in</strong> which participants were asked to rank the factors<br />
contribut<strong>in</strong>g to an improvement <strong>in</strong> food security. Factors coloured <strong>in</strong> blue were <strong>in</strong>fluenced by the<br />
project’s <strong>in</strong>terventions, while those <strong>in</strong> red were not: 62<br />
Figure 5: Factors Reportedly Contribut<strong>in</strong>g to Improved Food Security<br />
In this case, the programmes contributed to the use <strong>of</strong> improved seeds and fertilizer. However, they<br />
cannot claim responsibility for government extension services, and certa<strong>in</strong>ly not for improved ra<strong>in</strong>fall.<br />
Be aware <strong>of</strong> the biases that qualitative and quantitative research is susceptible to; for example,<br />
beneficiaries may wish to give a positive impression <strong>of</strong> the programme <strong>in</strong> order to please the<br />
<strong>in</strong>terviewer, or <strong>in</strong> order to ga<strong>in</strong> more fund<strong>in</strong>g. Likewise, the <strong>in</strong>terviewer may be biased towards their<br />
own organisation.<br />
F<strong>in</strong>ally, quantitative attribution analysis can be applied, <strong>in</strong>clud<strong>in</strong>g techniques like randomised control<br />
trials (RCTs), statistical analysis, or quasi-experimental techniques. They generally rely on collect<strong>in</strong>g<br />
quantitative data from large numbers <strong>of</strong> people, and us<strong>in</strong>g statistical techniques to create a<br />
‘counterfactual’, to understand what the situation would have been like without the <strong>in</strong>tervention. This is<br />
compared to the measured results <strong>of</strong> the <strong>in</strong>tervention to determ<strong>in</strong>e the improvement that can be<br />
attributed to the project. This is <strong>of</strong>ten seen as impractical <strong>in</strong> conflict affected environments, but it can be<br />
achieved.<br />
For example, the International Rescue Committee (IRC) is implement<strong>in</strong>g RCTs <strong>in</strong> Afghanistan, Burundi,<br />
DRC, and Liberia, <strong>in</strong>clud<strong>in</strong>g one <strong>of</strong> a sav<strong>in</strong>gs group programme. IRC’s experience suggests that RCTs are<br />
possible to implement <strong>in</strong> CAEs but face additional challenges. For <strong>in</strong>stance, the lack <strong>of</strong> population data<br />
62 Taken from Catley, Andrew et al, Participatory Impact Assessment - A Guide for Practitioners, Fe<strong>in</strong>ste<strong>in</strong><br />
International Centre, 2008, 50.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
35
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
means that select<strong>in</strong>g an appropriate sample can difficult. Census data must <strong>in</strong> some cases be collected<br />
directly. Further, randomization can <strong>in</strong>crease costs and complexity. In Liberia, for <strong>in</strong>stance, random<br />
selection <strong>of</strong> project sites requires the project to work across areas us<strong>in</strong>g two different languages and<br />
with significant distances between sites. This is particularly an issue when seek<strong>in</strong>g to measure changes <strong>in</strong><br />
conflict at a community level rather than an <strong>in</strong>dividual level: hav<strong>in</strong>g a sufficiently large sample for<br />
randomization would require work<strong>in</strong>g <strong>in</strong> several hundred villages. Another consideration <strong>in</strong> us<strong>in</strong>g a<br />
randomization approach <strong>in</strong> CAEs is that mak<strong>in</strong>g changes to project strategy – a more frequent necessity<br />
<strong>in</strong> such environments – can compromise the f<strong>in</strong>d<strong>in</strong>gs <strong>of</strong> the RCT. There is also a risk that randomization<br />
can exacerbate the conflict by creat<strong>in</strong>g resentment among control groups members <strong>of</strong> the treatment<br />
group. IRC’s experience suggests that this latter challenge can be overcome by be<strong>in</strong>g open and<br />
transparent about the process <strong>of</strong> select<strong>in</strong>g project clients. 63 Still, randomization approaches require<br />
highly tra<strong>in</strong>ed staff and high costs, and will not be realistic for most projects.<br />
In summary, there are several approaches to assess<strong>in</strong>g attribution. The most appropriate comb<strong>in</strong>ation <strong>of</strong><br />
approaches will vary based on the nature <strong>of</strong> the <strong>in</strong>tervention and capacity <strong>of</strong> the implement<strong>in</strong>g team.<br />
4.2 Key resources<br />
Mayne, John. Contribution Analysis: An approach to explor<strong>in</strong>g cause and effect, 2008.<br />
<strong>in</strong>troduction to Contribution Analysis.<br />
A short<br />
Catley, Andrew et al, Participatory Impact Assessment - A Guide for Practitioners, Fe<strong>in</strong>ste<strong>in</strong><br />
International Centre, 2008<br />
OECD. Evaluat<strong>in</strong>g Peacebuild<strong>in</strong>g Activities <strong>in</strong> Sett<strong>in</strong>gs <strong>of</strong> Conflict and Fragility: Improv<strong>in</strong>g Learn<strong>in</strong>g for<br />
Results. Provides guidance on assess<strong>in</strong>g attribution <strong>in</strong> the evaluation <strong>of</strong> peacebuild<strong>in</strong>g activities.<br />
Sen, Nabanita. Attribution: <strong>Measur<strong>in</strong>g</strong> Attributable Change Caused by a Programme. A brief,<br />
accessible outl<strong>in</strong>e <strong>of</strong> how to estimate attribution when apply<strong>in</strong>g the <strong>DCED</strong> Standard.<br />
Stern, Elliot et al. Broaden<strong>in</strong>g the Range <strong>of</strong> Designs and Methods for Impact Evaluations. Addresses<br />
the issue <strong>of</strong> attribution <strong>in</strong> depth, look<strong>in</strong>g at the role <strong>of</strong> mixed methods and theory-based approaches to<br />
assess<strong>in</strong>g attribution.<br />
5 Captur<strong>in</strong>g Wider Changes <strong>in</strong> the System or Market (Control Po<strong>in</strong>ts<br />
5.1)<br />
Systemic change refers to “[c]hanges <strong>in</strong> market systems and the structures, such as government and civil<br />
society, that support markets that cause susta<strong>in</strong>able shifts <strong>in</strong> the way those market systems and<br />
structures operate, for example, changes <strong>in</strong> relationships with<strong>in</strong> and among both private enterprises and<br />
public agencies, <strong>in</strong> <strong>in</strong>centives and <strong>in</strong> market support structures. Systemic change causes widespread<br />
<strong>in</strong>direct results such as crowd<strong>in</strong>g <strong>in</strong>, copy<strong>in</strong>g, enterprises shift<strong>in</strong>g sectors and changes <strong>in</strong> enterprise start-<br />
63 DFID, Interim Guidance Note: <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict-Affected States and<br />
Situations, Undated, 38.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
36
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
up and exit rates.” 64 <strong>Measur<strong>in</strong>g</strong> systemic change is extremely important to understand<strong>in</strong>g the full<br />
impacts <strong>of</strong> PSD programm<strong>in</strong>g. Its omission from results measurement frameworks means that projects<br />
will underreport their impact and miss many <strong>of</strong> the most important types <strong>of</strong> change.<br />
Ow<strong>in</strong>g to the limited experience <strong>in</strong> explicitly apply<strong>in</strong>g the entire <strong>DCED</strong> Standard <strong>in</strong> CAEs, there are still<br />
very few experiences <strong>in</strong> attempt<strong>in</strong>g to measure systemic change <strong>in</strong> a CAE context. Systemic change is<br />
less likely to occur where trust among neighbours and bus<strong>in</strong>esses has been weakened and economic<br />
activity is more limited. Estimations <strong>of</strong> the amount <strong>of</strong> copy<strong>in</strong>g and crowd<strong>in</strong>g-<strong>in</strong> sparked by project<br />
<strong>in</strong>terventions, particularly if derived from benchmarks <strong>in</strong> non-CAEs (e.g. economic multipliers) thus may<br />
need to be more conservative.<br />
6 Track<strong>in</strong>g Programme Costs (Control Po<strong>in</strong>ts 6.1 – 6.2)<br />
Track<strong>in</strong>g programm<strong>in</strong>g costs is important to understand and improve programm<strong>in</strong>g efficiency. In CAEs<br />
characterized by significant pressure to spend quickly, track<strong>in</strong>g costs is particularly important. However,<br />
projects <strong>in</strong> these contexts face a number <strong>of</strong> challenges. The costs <strong>of</strong> implementation are frequently<br />
higher yet achievements (e.g. employment generation, <strong>in</strong>creases <strong>in</strong> <strong>in</strong>come) fewer. 65 As such, be careful<br />
<strong>in</strong> benchmark<strong>in</strong>g the benefit-cost ratios <strong>of</strong> projects implemented <strong>in</strong> CAEs with those implemented<br />
elsewhere. Projects also need to be careful if publish<strong>in</strong>g f<strong>in</strong>ancial reports disaggregated by region or<br />
ethnic group. If spend<strong>in</strong>g is unequally distributed among conflict<strong>in</strong>g areas or groups, this may fuel<br />
tension and put project staff or partners at an <strong>in</strong>creased risk.<br />
7 Report<strong>in</strong>g Results (Control Po<strong>in</strong>ts 7.1 – 7.5)<br />
Report<strong>in</strong>g results creates transparency about development programm<strong>in</strong>g. Given the high visibility <strong>of</strong><br />
many CAEs and pressure from donors to show results, there is <strong>of</strong>ten great pressure to report results<br />
quickly. However, report<strong>in</strong>g project results can potentially create harm <strong>in</strong> CAEs. Identify<strong>in</strong>g specific<br />
clients creates the risk that they can be s<strong>in</strong>gled out for harm, even <strong>in</strong> seem<strong>in</strong>gly safe places like project<br />
reports.<br />
The pressure to report results can also have negative effects on the programme, encourag<strong>in</strong>g short term<br />
<strong>in</strong>terventions and ‘quick w<strong>in</strong>s’ at the expense <strong>of</strong> susta<strong>in</strong>able, longer-term change. Although this is<br />
unavoidable to some extent, programme staff should be aware <strong>of</strong> this pressure and manage<br />
relationships with donors to give themselves adequate time to achieve change <strong>in</strong> a CAE.<br />
Caution!<br />
Carefully consider how to safeguard monitor<strong>in</strong>g <strong>in</strong>formation and publish it <strong>in</strong> responsible ways that do<br />
not put others at risk.<br />
64 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />
<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 18.<br />
65 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />
Phase 1: Formative research and key issue diagnosis, undated, 5.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
37
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Consider the follow<strong>in</strong>g when safely report<strong>in</strong>g results:<br />
Data protection: Computers with sensitive data should be password protected, data encrypted<br />
and guarded. 66<br />
Pre-publish<strong>in</strong>g review: All monitor<strong>in</strong>g documentation should be reviewed with a conflictsensitive<br />
perspective prior to publication to ensure that it does not create the risk <strong>of</strong> harm.<br />
Strategies to achieve this can <strong>in</strong>clude seek<strong>in</strong>g permission from any person who is named <strong>in</strong> a<br />
report (<strong>in</strong>clud<strong>in</strong>g staff), develop<strong>in</strong>g guidel<strong>in</strong>es with key stakeholders (e.g. government if<br />
appropriate, donors, partners, local community), or ask<strong>in</strong>g these stakeholders to comment on<br />
drafts before review.<br />
Word choice: The language used by a project can itself be contentious. Us<strong>in</strong>g appropriate<br />
language (e.g. ‘context analysis’ <strong>in</strong>stead <strong>of</strong> conflict analysis, ‘tension’ <strong>in</strong>stead <strong>of</strong> conflict) can<br />
help. CAEs are <strong>of</strong>ten characterized by significant donor overlap. Multiple projects may work with<br />
the same partner bus<strong>in</strong>esses and households. Moreover, the results that you are try<strong>in</strong>g to<br />
achieve will be affected by many other actors, <strong>in</strong>clud<strong>in</strong>g government, local communities, and<br />
changes <strong>in</strong> the broader context. In such an environment, recognis<strong>in</strong>g the contributions <strong>of</strong> these<br />
other actors is essential.<br />
8 Manag<strong>in</strong>g the System for Results Measurement<br />
A successful results measurement system must be <strong>in</strong>tegrated <strong>in</strong>to all aspects <strong>of</strong> programme<br />
management, from design through implementation to monitor<strong>in</strong>g and evaluation. This section suggests<br />
ways <strong>in</strong> which the management system can be adjusted to CAEs.<br />
8.1 Establish<strong>in</strong>g a clear system for results measurement (Control Po<strong>in</strong>ts 8.1)<br />
Results measurement systems should use the follow<strong>in</strong>g approach to ease adoption.<br />
Invest from the beg<strong>in</strong>n<strong>in</strong>g. Many results measurement systems are added to exist<strong>in</strong>g projects after<br />
their launch, rather than be<strong>in</strong>g designed from <strong>in</strong>ception. This <strong>of</strong>ten means it is too late to generate<br />
quality data – for example, because basel<strong>in</strong>e data is not collected – and creates additional<br />
challenges for staff and partners who must adapt to a new system. It is worth <strong>in</strong>vest<strong>in</strong>g additional<br />
resources dur<strong>in</strong>g the design stage to ensure that the results measurement system is robust from the<br />
beg<strong>in</strong>n<strong>in</strong>g.<br />
Phase <strong>in</strong> the monitor<strong>in</strong>g system. The <strong>DCED</strong> Standard can be demand<strong>in</strong>g, especially for new<br />
partners. Consider phas<strong>in</strong>g <strong>in</strong> the results measurement system, successfully implement<strong>in</strong>g one<br />
component before mov<strong>in</strong>g on to the next. This <strong>in</strong>creases the likelihood <strong>of</strong> the results measurement<br />
system be<strong>in</strong>g used for project management. For example, you may <strong>in</strong>itially require new partners to<br />
monitor and report only aga<strong>in</strong>st activities. Once that is established, move on to monitor<strong>in</strong>g outputs,<br />
then outcomes. At each stage, give further tra<strong>in</strong><strong>in</strong>g, feedback and encouragement. Ensure that you<br />
have matched the complexity <strong>of</strong> the system with the ability <strong>of</strong> your partners, and have a clear plan<br />
for mov<strong>in</strong>g towards implementation <strong>of</strong> the full Standard as soon as possible. However, make sure<br />
66 MEDA, Monday <strong>Development</strong> Magaz<strong>in</strong>e, July 2010, 14.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
38
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
that partners are aware <strong>of</strong> this plan, so that they understand and can prepare for the change <strong>in</strong> the<br />
system.<br />
Require more regular report<strong>in</strong>g. Shorter report<strong>in</strong>g periods may be appropriate <strong>in</strong> more difficult<br />
environments. A DFID-funded programme <strong>in</strong> the DRC changed from six monthly to quarterly<br />
progress report<strong>in</strong>g. It also adopted monthly targets to support a more flexible management<br />
approach <strong>in</strong> the early stages <strong>of</strong> the project. 67 This is particularly effective if it is tied to opportunities<br />
for the project to revise its results cha<strong>in</strong>s and <strong>in</strong>dicators if necessary, and based on a realistic<br />
assessment <strong>of</strong> what the programme can provide.<br />
Knowledge management. Given the frequent high turnover <strong>of</strong> staff, it is essential to properly<br />
document the results management system. This should <strong>in</strong>clude a clear explanation <strong>of</strong> the results<br />
cha<strong>in</strong>, monitor<strong>in</strong>g plan, and systems for collect<strong>in</strong>g and analys<strong>in</strong>g results. It should also cover key<br />
decisions and compromises that were made. For example, if the results cha<strong>in</strong> was simplified to<br />
improve usability, document the changes that were made and the reasons beh<strong>in</strong>d them. This will<br />
prevent future managers from redo<strong>in</strong>g the same work.<br />
8.2 Human and f<strong>in</strong>ancial resources (Control Po<strong>in</strong>t 8.2)<br />
Recruit<strong>in</strong>g and reta<strong>in</strong><strong>in</strong>g qualified staff is a serious challenge <strong>in</strong> CAEs where local capacity may be low,<br />
<strong>in</strong>ternational staff unwill<strong>in</strong>g to work and turnover extremely high.<br />
Consider the composition <strong>of</strong> your team. Are they all from the same region or ethnic group<br />
International organisations commonly hire staff with a m<strong>in</strong>imum level <strong>of</strong> education – who may<br />
primarily be from historically dom<strong>in</strong>ant groups. This may reduce the perceived neutrality <strong>of</strong> your<br />
<strong>in</strong>tervention, or impede cooperation from local communities and government.<br />
Ensure that the team are well <strong>in</strong>formed about conflict, and conflict sensitivity. Try to hire staff with<br />
experience work<strong>in</strong>g <strong>in</strong> conflicts, and deliver additional tra<strong>in</strong><strong>in</strong>g on conflict sensitivity as necessary.<br />
Staff may be <strong>in</strong>itially reluctant to report on conflict, and may need additional tra<strong>in</strong><strong>in</strong>g to encourage<br />
them to do so. It may also be necessary to hire staff focused specifically on conflict sensitivity.<br />
Invest <strong>in</strong> tra<strong>in</strong><strong>in</strong>g courses, mentor<strong>in</strong>g and additional support for local staff. Give regular tra<strong>in</strong><strong>in</strong>g,<br />
opportunities for staff to share lessons, and constructive feedback on the monitor<strong>in</strong>g <strong>in</strong>formation<br />
that they provide.<br />
Tra<strong>in</strong> local partners or communities to assume additional M&E responsibilities. The Aga Khan<br />
<strong>Development</strong> Network, for example, has supported Social Audit Committees, elected by the village<br />
to scrut<strong>in</strong>ize the village council accounts. Aga Khan tra<strong>in</strong>s the committees on basic book-keep<strong>in</strong>g,<br />
supports them to exam<strong>in</strong>e the village council accounts, and then holds open village-wide assemblies<br />
where they present their f<strong>in</strong>d<strong>in</strong>gs. 68<br />
Decentralize your monitor<strong>in</strong>g. Staff that are close to the ground are <strong>of</strong>ten better able to adjust <strong>in</strong><br />
conflict situations and also typically less expensive than adopt<strong>in</strong>g a centralized monitor<strong>in</strong>g strategy.<br />
67 Cox, Marcus and Nigel Thornton, Manag<strong>in</strong>g Results <strong>in</strong> CAEs: A stock-take <strong>of</strong> lessons, experience and practice,<br />
2010, 20.<br />
68 C<strong>of</strong>fey International <strong>Development</strong>, The Cost <strong>of</strong> a Good Deed - Monitor<strong>in</strong>g <strong>in</strong> Post-Conflict Environments and the<br />
Application <strong>of</strong> the Logframe, 2009.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
39
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Hire regional consultants. They are generally less expensive than <strong>in</strong>ternational consultants, may<br />
speak the language, have local knowledge, and may be more acceptable to the local community.<br />
This strategy depends on the availability <strong>of</strong> regional experts and thus will be more applicable <strong>in</strong><br />
some contexts than others.<br />
Case Study: Build<strong>in</strong>g Local Capacity <strong>in</strong> Results Measurement <strong>in</strong> Afghanistan 69<br />
The International Fund for Agricultural <strong>Development</strong> (IFAD) Afghanistan builds local capacity<br />
through the Young Pr<strong>of</strong>essionals Programme. They hire recent agriculture sciences graduates, who<br />
have a strong academic background but little work experience. These young pr<strong>of</strong>essionals work for a<br />
year with IFAD’s local partners, conduct<strong>in</strong>g field surveys and site visits <strong>in</strong> order to <strong>in</strong>dependently<br />
verify outputs and results.<br />
The programme views this strategy as creat<strong>in</strong>g multiple benefits. It is able to cheaply employ<br />
motivated, energetic young pr<strong>of</strong>essionals, enabl<strong>in</strong>g effective monitor<strong>in</strong>g <strong>of</strong> remote projects. In<br />
return, the young pr<strong>of</strong>essionals receive a stipend and ga<strong>in</strong> <strong>in</strong>valuable work experience. Follow<strong>in</strong>g the<br />
pilot, 80% <strong>of</strong> participat<strong>in</strong>g graduates found a job <strong>in</strong> their desired fields.<br />
Literature typically suggests devot<strong>in</strong>g 5-10% <strong>of</strong> a project’s budget to results measurement, which is likely<br />
to be more costly <strong>in</strong> CAEs. 70 In particular, there is likely to be a greater need for primary data collection,<br />
due to non-existent or poor quality secondary data. Changes <strong>in</strong> results cha<strong>in</strong>s and <strong>in</strong>dicators created by<br />
shifts <strong>in</strong> the environment will require additional data collection to re-establish basel<strong>in</strong>es. Staff, transport<br />
and accommodation are also frequently more expensive.<br />
In environments with multiple donors, look for opportunities to share results measurement costs. For<br />
example, multi-donor evaluations may be able to attribute change more effectively. They br<strong>in</strong>g together<br />
different viewpo<strong>in</strong>ts, and reduce the cost both to the commission<strong>in</strong>g partner and the recipient<br />
stakeholders. If it is impossible to dist<strong>in</strong>guish between the effects <strong>of</strong> different donor efforts, it may be<br />
more realistic to evaluate the donor effort as a whole. However, this can also br<strong>in</strong>g additional logistical<br />
and management challenges, as it may require coord<strong>in</strong>ation between donors with different political<br />
agendas and theories <strong>of</strong> change. 71<br />
8.3 Key resources<br />
The People <strong>in</strong> Aid Code is an <strong>in</strong>ternationally recognised management tool that aims to help<br />
humanitarian and development organisations improve the quality <strong>of</strong> their human resources<br />
management.<br />
69 Interview with Abdul Latif Zahed, IFAD Afghanistan, September 17, 2012.<br />
70 Church and Roberts. Design<strong>in</strong>g for Results, Integrat<strong>in</strong>g Monitor<strong>in</strong>g and Evaluation <strong>in</strong>to Conflict Transformation<br />
Programmes, 2006, 132; DFID. <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict Affected States and<br />
Situation, 18; Dalberg, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries, Phase 1: Formative<br />
research and key issue diagnosis, 7.<br />
71 OECD, Encourag<strong>in</strong>g Effective Evaluation <strong>of</strong> Conflict Prevention and Peacebuild<strong>in</strong>g Activities, 2007.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
40
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Annex 1: Key Def<strong>in</strong>itions 72<br />
Activity: A discrete piece <strong>of</strong> work, typically represented by a contract between the programme and a<br />
contractor, partner or consultant. Interventions typically consist <strong>of</strong> several activities, that are <strong>in</strong>tended<br />
to achieve change at various different po<strong>in</strong>ts <strong>in</strong> the overall market system.<br />
Assumption: A supposition or best guess which forms part <strong>of</strong> the basis for calculation <strong>of</strong> an <strong>in</strong>dicator<br />
value.<br />
Attribution: The ascription <strong>of</strong> a causal l<strong>in</strong>k between observed (or expected to be observed) changes and<br />
a specific <strong>in</strong>tervention<br />
Basel<strong>in</strong>e: An analysis describ<strong>in</strong>g the situation prior to a development <strong>in</strong>tervention, aga<strong>in</strong>st which<br />
progress can be assessed or comparisons made.<br />
Conflict: The result <strong>of</strong> parties disagree<strong>in</strong>g (e.g. about the distribution <strong>of</strong> material or symbolic resources)<br />
and act<strong>in</strong>g on the basis <strong>of</strong> these perceived <strong>in</strong>compatibilities. 73<br />
Conflict affected environments: Environments that have recently been, currently are or are prone to<br />
be<strong>in</strong>g affected by conflict.<br />
Conflict analysis: The systematic study <strong>of</strong> the pr<strong>of</strong>ile, causes, actors, and dynamics <strong>of</strong> conflict. 74<br />
Conflict sensitivity: The ability <strong>of</strong> an organisation to understand the context <strong>in</strong> which it operates;<br />
understand the <strong>in</strong>teraction between an <strong>in</strong>tervention and the context; and act upon the understand<strong>in</strong>g <strong>of</strong><br />
this <strong>in</strong>teraction, <strong>in</strong> order to avoid negative impacts and maximise positive impacts. 75<br />
Copy<strong>in</strong>g: Other target enterprises copy<strong>in</strong>g behavioural changes that those affected directly by<br />
programme activities have adopted. 76<br />
Counterfactual: The situation or condition which hypothetically may prevail for <strong>in</strong>dividuals,<br />
organizations, or groups were there no development <strong>in</strong>tervention.<br />
Crowd<strong>in</strong>g <strong>in</strong>: Enterprises at levels other than the target level copy<strong>in</strong>g behaviours that those affected by<br />
programme activities have adopted or enter<strong>in</strong>g a sector or value cha<strong>in</strong> as a result <strong>of</strong> improved <strong>in</strong>centives<br />
and environment created (at least partly) by the programme. This term also applies to government<br />
agencies or civil society organizations, who are not directly <strong>in</strong>volved <strong>in</strong> the programme, copy<strong>in</strong>g<br />
72 Unless noted, all def<strong>in</strong>itions drawn from OECD, Glossary <strong>of</strong> Key Terms <strong>in</strong> Evaluation and Results Based<br />
Management, 2002.<br />
73 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack, 3.<br />
74 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack,<br />
chapter 2, page 1<br />
75 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack, 3.<br />
76 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />
<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 15.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
41
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
behaviours <strong>of</strong> those who are directly <strong>in</strong>volved <strong>in</strong> the programme, or who change their behaviour as a<br />
result <strong>of</strong> improved <strong>in</strong>centives or environment created (at least partly) by the programme.. 77<br />
Displacement: Some enterprises may be negatively affected because others are benefit<strong>in</strong>g from<br />
programme activities. Displacement is the amount <strong>of</strong> negative effect on those enterprises harmed by<br />
programme activities. 78<br />
Do No Harm: Ways <strong>in</strong> which <strong>in</strong>ternational humanitarian and development assistance given <strong>in</strong> conflict<br />
sett<strong>in</strong>gs may be provided so that, rather than exacerbat<strong>in</strong>g and worsen<strong>in</strong>g the conflict, it helps local<br />
people disengage from fight<strong>in</strong>g and develop systems for settl<strong>in</strong>g the problems which prompt conflict<br />
with<strong>in</strong> their societies.<br />
Evaluation: The systematic and objective assessment <strong>of</strong> an on-go<strong>in</strong>g or completed project, programme<br />
or policy, its design, implementation and results. Evaluation also refers to the process <strong>of</strong> determ<strong>in</strong><strong>in</strong>g<br />
the worth or significance <strong>of</strong> an activity, policy or program. An assessment, as systematic and objective as<br />
possible, <strong>of</strong> a planned, on-go<strong>in</strong>g, or completed development <strong>in</strong>tervention.<br />
Impacts: Positive and negative, primary and secondary long-term effects produced by a development<br />
<strong>in</strong>tervention, directly or <strong>in</strong>directly, <strong>in</strong>tended or un<strong>in</strong>tended.<br />
Intervention: A coherent set <strong>of</strong> activities that share a s<strong>in</strong>gle results cha<strong>in</strong>, and are designed to achieve a<br />
specific and limited change. 79 A project usually manages multiple <strong>in</strong>terventions and thus multiple results<br />
cha<strong>in</strong>s.<br />
Monitor<strong>in</strong>g: A cont<strong>in</strong>u<strong>in</strong>g function that uses systematic collection <strong>of</strong> data on specified <strong>in</strong>dicators to<br />
provide management and the ma<strong>in</strong> stakeholders <strong>of</strong> an ongo<strong>in</strong>g development <strong>in</strong>tervention with<br />
<strong>in</strong>dications <strong>of</strong> the extent <strong>of</strong> progress and achievement <strong>of</strong> objectives and progress <strong>in</strong> the use <strong>of</strong> allocated<br />
funds.<br />
Peacebuild<strong>in</strong>g: Measures designed to consolidate peaceful relations and strengthen viable political,<br />
socio-economic and cultural <strong>in</strong>stitutions capable <strong>of</strong> mediat<strong>in</strong>g conflict, and to strengthen other<br />
mechanisms that will either create or support the necessary conditions for susta<strong>in</strong>ed peace. 80<br />
Proxy <strong>in</strong>dicator: An <strong>in</strong>dicator for which measurable change is clearly and reliably correlated with an<br />
<strong>in</strong>dicator <strong>of</strong> a change that the programme aims to achieve (but is generally more practical to measure). 81<br />
77 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />
<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 15.<br />
78 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />
<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 16.<br />
79 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />
<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 16.<br />
80 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack, 5.<br />
81 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />
<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 17.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
42
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Results cha<strong>in</strong>: The causal sequence for a development <strong>in</strong>tervention that stipulates the necessary<br />
sequence to achieve desired objectives beg<strong>in</strong>n<strong>in</strong>g with <strong>in</strong>puts, mov<strong>in</strong>g through activities and outputs,<br />
and culm<strong>in</strong>at<strong>in</strong>g <strong>in</strong> outcomes, impacts and feedback.<br />
Results measurement: The process <strong>of</strong> design<strong>in</strong>g a measurement system <strong>in</strong> order to estimate a<br />
programme’s impact so that it can be used to report results and improve project management. 82<br />
Susta<strong>in</strong>ability: The cont<strong>in</strong>uation <strong>of</strong> benefits from a development <strong>in</strong>tervention after major development<br />
assistance has been completed. The probability <strong>of</strong> cont<strong>in</strong>ued long term benefits.<br />
Systemic change: Change <strong>in</strong> systems that are caused by <strong>in</strong>troduc<strong>in</strong>g alternative <strong>in</strong>novative susta<strong>in</strong>able<br />
bus<strong>in</strong>ess models at support market level (such as <strong>in</strong> private sector, government, civil society, public<br />
policy level). These changes <strong>of</strong>ten cause widespread <strong>in</strong>direct impact by crowd<strong>in</strong>g <strong>in</strong> at support market<br />
levels impact and copy<strong>in</strong>g at f<strong>in</strong>al beneficiary level. 83<br />
Theory <strong>of</strong> Change: A testable hypothesis regard<strong>in</strong>g how the planned activities will contribute to<br />
achiev<strong>in</strong>g the desired results for the programme. 84<br />
Triangulation: The use <strong>of</strong> multiple theories, sources or types <strong>of</strong> <strong>in</strong>formation, or types <strong>of</strong> analysis to<br />
verify and substantiate an assessment.<br />
82 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />
<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 17.<br />
83 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />
<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 18.<br />
84 CARE, International UK, Peacebuild<strong>in</strong>g with Impact: Def<strong>in</strong><strong>in</strong>g Theories <strong>of</strong> Change, 2012, 3.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
43
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Annex 2: Acronyms and Abbreviations<br />
BMO Bus<strong>in</strong>ess Membership Organization<br />
CAE<br />
<strong>DCED</strong><br />
DFID<br />
IFAD<br />
IFC<br />
IRC<br />
MSA<br />
M&E<br />
NGO<br />
OECD<br />
RCT<br />
Conflict Affected Environment<br />
Donor Committee for Enterprise <strong>Development</strong><br />
Department for International <strong>Development</strong><br />
International Fund for Agricultural <strong>Development</strong><br />
International F<strong>in</strong>ance Corporation<br />
International Research Committee<br />
MarketShare Associates<br />
Monitor<strong>in</strong>g and Evaluation<br />
Non-Governmental Organization<br />
Organisation for Economic Co-operation and <strong>Development</strong><br />
Randomized Control Trial<br />
UNDP United Nations <strong>Development</strong> Programme<br />
USAID United States Agency for International <strong>Development</strong><br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
44
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Annex 3:<br />
Overall Resource List<br />
Onl<strong>in</strong>e Resources<br />
<strong>DCED</strong> <strong>Measur<strong>in</strong>g</strong> Results Resources. Provides several key resources for results measur<strong>in</strong>g <strong>of</strong> PSD and<br />
peacebuild<strong>in</strong>g.<br />
<strong>DCED</strong> Onl<strong>in</strong>e Library on PSD <strong>in</strong> Conflict-Affected Environments. Provides access to more than 450<br />
documents on the theme.<br />
<strong>DCED</strong> Compilation <strong>of</strong> Guidance by Various Agencies for Practitioners <strong>of</strong> PSD <strong>in</strong> Conflict-Affected<br />
Environments. L<strong>in</strong>ks to exist<strong>in</strong>g advice and guidance for assessment, programme design and<br />
implementation and measur<strong>in</strong>g results.<br />
Design, Monitor<strong>in</strong>g and Evaluation for Peacebuild<strong>in</strong>g. Offers resources and discussions on resources<br />
for monitor<strong>in</strong>g and evaluation <strong>of</strong> peacebuild<strong>in</strong>g programm<strong>in</strong>g.<br />
Organization for Economic Co-operation and <strong>Development</strong>. Overview <strong>of</strong> the Network's Current Work<br />
On Evaluat<strong>in</strong>g Conflict Prevention And Peacebuild<strong>in</strong>g. Presents the Organization for Economic Cooperation<br />
and <strong>Development</strong> (OECD)’s conflict prevention and peacebuild<strong>in</strong>g evaluations.<br />
Conflict Analysis Guidel<strong>in</strong>es<br />
Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource<br />
Pack, Chapter Two. Very comprehensive conflict sensitivity guidel<strong>in</strong>es. Chapter two describes conflict<br />
analysis <strong>in</strong> detail. Of particular <strong>in</strong>terest are pages 12-40 <strong>in</strong> chapter two, which summarise 15 different<br />
conflict analysis tools from major donors. For each they discuss the primary purpose, summarise the<br />
ma<strong>in</strong> steps, and give examples <strong>of</strong> lessons learned and current applications. If you are select<strong>in</strong>g a conflict<br />
analysis framework, this is an <strong>in</strong>valuable resource.<br />
Collaborative for <strong>Development</strong> Action, Inc. The Do No Harm Handbook: The Framework for Analyz<strong>in</strong>g<br />
the Impact <strong>of</strong> Assistance on Conflict. This guide provides a step-by-step guide to assess<strong>in</strong>g the potential<br />
<strong>of</strong> a project to cause harm <strong>in</strong> its selected context.<br />
GIZ. Peace and Conflict Assessment (PCA): A Methodological Framework for the Conflict- and Peace-<br />
Oriented Alignment <strong>of</strong> <strong>Development</strong> Programmes. Exam<strong>in</strong>es the application <strong>of</strong> peace and conflict<br />
assessments at multiple stages <strong>in</strong> a project, <strong>in</strong>clud<strong>in</strong>g impact monitor<strong>in</strong>g.<br />
USAID. Conflict Assessment Framework. Version 2.0. Provides a framework for diagnos<strong>in</strong>g a conflict<br />
and generat<strong>in</strong>g responses. Application guidel<strong>in</strong>es can be found here.<br />
Results Cha<strong>in</strong>s and Theories <strong>of</strong> Change <strong>in</strong> CAEs<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
45
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Becker, Sab<strong>in</strong>e. Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes. Reviews<br />
many <strong>of</strong> the key relationships between PSD and peacebuild<strong>in</strong>g.<br />
CARE International UK. Peacebuild<strong>in</strong>g with Impact: Def<strong>in</strong><strong>in</strong>g Theories <strong>of</strong> Change. Offers guidance <strong>in</strong><br />
develop<strong>in</strong>g the logic <strong>of</strong> how project <strong>in</strong>terventions will impact upon peacebuild<strong>in</strong>g.<br />
Davis, Peter. Corporations, Global Governance and Post-Conflict Reconstruction. Synthesizes the<br />
impacts corporations may have on a conflict and support<strong>in</strong>g reconstruction, draw<strong>in</strong>g from Azerbaijan,<br />
Bosnia and Rwanda.<br />
Humphreys, Macartan. Economics and Violent Conflict. Reviews the evidence for l<strong>in</strong>kages between<br />
economics and conflict.<br />
Mercy Corps. Peacebuild<strong>in</strong>g through Economic <strong>Development</strong> Approach. This publication outl<strong>in</strong>es five<br />
theories <strong>of</strong> change that Mercy Corps is implement<strong>in</strong>g and test<strong>in</strong>g through its work and provides<br />
examples <strong>of</strong> each.<br />
Mierke, Axel. Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes. Analyzes the<br />
potential support <strong>of</strong> private sector development programm<strong>in</strong>g to peacebuild<strong>in</strong>g.<br />
Pottebaum, David. Relationships between Conflict, Poverty, Inequality, and Economic Growth. This<br />
publication reviews the evidence for relationships between conflict, poverty, <strong>in</strong>equality, and economic<br />
growth.<br />
Sen, Nabanita. A Guide to Mak<strong>in</strong>g Results Cha<strong>in</strong>s. Provides the key steps required to create a results<br />
cha<strong>in</strong>.<br />
USAID. Conflict-Sensitive Approaches to Value Cha<strong>in</strong> <strong>Development</strong>, microNote #101. Outl<strong>in</strong>es how to<br />
<strong>in</strong>corporate a conflict lens when conduct<strong>in</strong>g research on value cha<strong>in</strong>s that can <strong>in</strong>form the results cha<strong>in</strong><br />
logic.<br />
Vogel, Isabel. Review <strong>of</strong> the uses <strong>of</strong> ‘Theory <strong>of</strong> Change’ <strong>in</strong> International <strong>Development</strong>. This report<br />
exam<strong>in</strong>es how theories <strong>of</strong> change are used <strong>in</strong> the field, different applications and best practice.<br />
Conflict Sensitivity<br />
Conflict Sensitivity Consortium. Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian<br />
Assistance and Peace Build<strong>in</strong>g Resource Pack. Provides guidance on how to <strong>in</strong>corporate a conflict<br />
sensitive approach throughout the project cycle, <strong>in</strong>clud<strong>in</strong>g monitor<strong>in</strong>g and evaluation.<br />
Conflict Sensitivity Consortium. A How To Guide to Conflict Sensitivity. Based on the above<br />
document, this adopts a similar approach. It is updated and shortened slightly from the orig<strong>in</strong>al resource<br />
pack.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
46
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
Def<strong>in</strong><strong>in</strong>g Indicators<br />
Agoglia et al. <strong>Measur<strong>in</strong>g</strong> Progress <strong>in</strong> Conflict Environments. Provides a significant volume <strong>of</strong> outcome<br />
<strong>in</strong>dicators around five aspects <strong>of</strong> a conflict, <strong>in</strong>clud<strong>in</strong>g the existence <strong>of</strong> a susta<strong>in</strong>able economy. These are<br />
high-level <strong>in</strong>dicators for gaug<strong>in</strong>g the overall stability <strong>of</strong> a situation, rather than the direct success <strong>of</strong> an<br />
<strong>in</strong>dividual project<br />
CDA. Indications for Assess<strong>in</strong>g Aid’s Impact on Conflict. Provides various <strong>in</strong>dications <strong>of</strong> a project’s<br />
positive and negative effects on the conflict.<br />
Church, Cheyanne and Mark M. Roberts. Design<strong>in</strong>g for Results: Integrat<strong>in</strong>g M&E <strong>in</strong> Conflict<br />
Transformation Programmes. General guidel<strong>in</strong>es on monitor<strong>in</strong>g and evaluat<strong>in</strong>g <strong>in</strong> CAEs. Chapter four<br />
<strong>in</strong>cludes a good discussion on <strong>in</strong>dicators and some helpful examples.<br />
DFID. Interim Guidance Note: <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict-Affected<br />
States and Situations. Annex A lists <strong>of</strong> <strong>in</strong>dicators for high level objectives, and possible data sources.<br />
More focused on country programmes than projects.<br />
Mercy Corps. Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict & Economics:<br />
Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Provides a menu <strong>of</strong> 29 peace and conflict <strong>in</strong>dicators and 49<br />
economics and conflict <strong>in</strong>dicators. Each <strong>in</strong>dicator is def<strong>in</strong>ed and a list <strong>of</strong> potential disaggregations is<br />
suggested. There is a useful discussion <strong>of</strong> how the <strong>in</strong>dicators were selected and measured, <strong>in</strong>clud<strong>in</strong>g the<br />
tools and surveys that were used.<br />
Indicator Measurement <strong>in</strong> CAEs<br />
Mercy Corps. Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions: Conflict & Economics:<br />
Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Provides several conflict monitor<strong>in</strong>g tools, <strong>in</strong>clud<strong>in</strong>g a violent<br />
<strong>in</strong>cident report<strong>in</strong>g form, a disputes and dispute resolution assessment, a scored community relationship<br />
mapp<strong>in</strong>g tool, and a conflict and resource mapp<strong>in</strong>g tool.<br />
Catley, Burns, Abebe and Suji, Participatory Impact Assessment: A Guide for Practitioners, 2008,<br />
Fe<strong>in</strong>ste<strong>in</strong> Centre. Outl<strong>in</strong>es an eight-step approach to conduct<strong>in</strong>g a participatory impact assessment.<br />
Frontl<strong>in</strong>e SMS. Frontl<strong>in</strong>e SMS is a free s<strong>of</strong>tware for creat<strong>in</strong>g and manag<strong>in</strong>g SMS activities such as<br />
mak<strong>in</strong>g announcements, conduct<strong>in</strong>g polls, and automatically reply<strong>in</strong>g to <strong>in</strong>com<strong>in</strong>g SMS. In particular, it is<br />
excellent for manag<strong>in</strong>g large mobile phone surveys, and was used <strong>in</strong> the above example by Colombia<br />
University to gather data <strong>in</strong> Eastern DRC.<br />
Muaz Jalil. Practical Guidel<strong>in</strong>es for Conduct<strong>in</strong>g Research. Provides a review <strong>of</strong> good practices <strong>in</strong><br />
design<strong>in</strong>g and conduct<strong>in</strong>g research <strong>in</strong> l<strong>in</strong>e with the <strong>DCED</strong> Standard.<br />
Sartorius, Rolf and Christopher Carver. Monitor<strong>in</strong>g, Evaluation and Learn<strong>in</strong>g for Fragile States and<br />
Peacebuild<strong>in</strong>g Programs: Practical Tools for Improv<strong>in</strong>g Program Performance and Results. This<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
47
<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />
provides a large number <strong>of</strong> tools for measur<strong>in</strong>g peacebuild<strong>in</strong>g programmes, alongside tips on<br />
measurement and example <strong>in</strong>dicators.<br />
Attribution <strong>in</strong> CAEs<br />
Evaluation Journal. Special Issue: Contribution analysis. Multiple articles <strong>in</strong>troduc<strong>in</strong>g the concept <strong>of</strong><br />
contribution analysis and its application to results measurement.<br />
Mayne, John. Contribution Analysis: An approach to explor<strong>in</strong>g cause and effect, 2008. A short<br />
<strong>in</strong>troduction to Contribution Analysis.<br />
OECD. Evaluat<strong>in</strong>g Peacebuild<strong>in</strong>g Activities <strong>in</strong> Sett<strong>in</strong>gs <strong>of</strong> Conflict and Fragility: Improv<strong>in</strong>g Learn<strong>in</strong>g for<br />
Results. Provides guidance on assess<strong>in</strong>g attribution <strong>in</strong> the evaluation <strong>of</strong> peacebuild<strong>in</strong>g activities.<br />
Sen, Nabanita. Attribution: <strong>Measur<strong>in</strong>g</strong> Attributable Change Caused by a Programme. A brief,<br />
accessible outl<strong>in</strong>e <strong>of</strong> how to estimate attribution when apply<strong>in</strong>g the <strong>DCED</strong> Standard. A brief, accessible<br />
outl<strong>in</strong>e <strong>of</strong> how to estimate attribution when apply<strong>in</strong>g the <strong>DCED</strong> Standard.<br />
Stern, Elliot et al. Broaden<strong>in</strong>g the Range <strong>of</strong> Designs and Methods for Impact Evaluations. Addresses<br />
the issue <strong>of</strong> attribution <strong>in</strong> depth, look<strong>in</strong>g at the role <strong>of</strong> mixed methods and theory-based approaches to<br />
assess<strong>in</strong>g attribution.<br />
System Management <strong>in</strong> CAEs<br />
The People <strong>in</strong> Aid Code is an <strong>in</strong>ternationally recognised management tool that aims to help<br />
humanitarian and development organisations improve the quality <strong>of</strong> their human resources<br />
management.<br />
Evaluation<br />
OECD. Evaluat<strong>in</strong>g Peacebuild<strong>in</strong>g Activities <strong>in</strong> Sett<strong>in</strong>gs <strong>of</strong> Conflict and Fragility: Improv<strong>in</strong>g Learn<strong>in</strong>g for<br />
Results. Provides guidance on how to conduct evaluations <strong>in</strong> CAEs.<br />
Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />
48