15.01.2015 Views

Measuring Achievements of Private Sector Development in ... - DCED

Measuring Achievements of Private Sector Development in ... - DCED

Measuring Achievements of Private Sector Development in ... - DCED

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong><br />

<strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Practical Guidel<strong>in</strong>es<br />

for Implement<strong>in</strong>g the <strong>DCED</strong> Standard<br />

Version 2, July 2013<br />

By Ben Fowler and Adam Kessler<br />

MarketShare Associates


These guidel<strong>in</strong>es have been produced as a companion to the Donor Committee for Enterprise <strong>Development</strong><br />

(<strong>DCED</strong>) Standard for Results Measurement (hereafter referred to as the <strong>DCED</strong> Standard) and associated<br />

guidance documents.<br />

The guidel<strong>in</strong>es are based on extensive <strong>in</strong>terviews with experts and field practitioners, desk research, and<br />

two cases studies based on country visits which analysed <strong>in</strong> depth programmes’ results measurement<br />

experiences <strong>in</strong> conflict-affected environments: The DFID-funded Susta<strong>in</strong>able Employment and Economic<br />

<strong>Development</strong> (SEED) programme <strong>in</strong> Somalia and the BMZ-funded Youth Employment programme<br />

implemented by GIZ <strong>in</strong> Sierra Leone.<br />

The guidel<strong>in</strong>es have been commissioned by the <strong>DCED</strong> Work<strong>in</strong>g Group on <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> (PSD) <strong>in</strong><br />

Conflict-Affected Environments (CAEs). It was produced by MarketShare Associates (MSA) and was written by<br />

Ben Fowler and Adam Kessler. Feedback is welcome and should be sent to ben@marketshareassociates.com<br />

and adam@kessler.co.uk.<br />

For more <strong>in</strong>formation on the <strong>DCED</strong> Standard for Results Measurement, please visit the <strong>DCED</strong> website at<br />

www.enterprise-development.org/page/measur<strong>in</strong>g-and-report<strong>in</strong>g-results or contact the <strong>DCED</strong> at<br />

Coord<strong>in</strong>ator@Enterprise-<strong>Development</strong>.org.<br />

To view the <strong>DCED</strong> Knowledge Page on PSD <strong>in</strong> CAEs, <strong>in</strong>clud<strong>in</strong>g an onl<strong>in</strong>e library with more than 450<br />

documents, and guidance by various agencies, please go to www.enterprise-development.org/page/cae.<br />

For more <strong>in</strong>formation about the activities <strong>of</strong> the <strong>DCED</strong> Work<strong>in</strong>g Group on PSD <strong>in</strong> CAEs, please refer to<br />

http://www.enterprise-development.org/page/caesg.<br />

Front page photo credits (from left to right): morguefile.com; Rebuild<strong>in</strong>g <strong>of</strong> a road and bridge <strong>in</strong> Tajikistan, by<br />

Stefan Erber, GIZ; Vocational tra<strong>in</strong><strong>in</strong>g <strong>of</strong> former child soldiers who benefitted from ILO’s International<br />

Programme on the Elim<strong>in</strong>ation <strong>of</strong> Child Labour (IPEC) <strong>in</strong> DRC, © International Labour Organization/ Marcel<br />

Crozet; UN cars and cattle dealer <strong>in</strong> Afghanistan, by Jochem Theis, GIZ; Survey for ILO’s JobNet Project <strong>in</strong> Sri<br />

Lanka, © International Labour Organization/ Marcel Crozet.


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Table <strong>of</strong> Contents<br />

I. Introduction .......................................................................................................................................... 3<br />

II. How to Use these Guidel<strong>in</strong>es ............................................................................................................ 3<br />

III. If You Are New to the <strong>DCED</strong> Standard .................................................................................................. 4<br />

IV. Key Pr<strong>in</strong>ciples to Implement<strong>in</strong>g the <strong>DCED</strong> Standard <strong>in</strong> CAEs ................................................................ 5<br />

1 Articulat<strong>in</strong>g Results Cha<strong>in</strong>s .................................................................................................................... 6<br />

1.1 Incorporat<strong>in</strong>g conflict <strong>in</strong>to results cha<strong>in</strong>s (Control Po<strong>in</strong>t 1.1-1.2) ................................................ 7<br />

1.2 Manag<strong>in</strong>g the results cha<strong>in</strong> (Control Po<strong>in</strong>ts 1.3 – 1.4) ................................................................ 18<br />

1.3 Key resources .............................................................................................................................. 19<br />

2 Def<strong>in</strong><strong>in</strong>g Indicators <strong>of</strong> Change ............................................................................................................. 20<br />

2.1 Def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators for results cha<strong>in</strong>s (Control Po<strong>in</strong>t 2.1) ........................................................... 21<br />

2.2 <strong>Measur<strong>in</strong>g</strong> the universal impact <strong>in</strong>dicators (Control Po<strong>in</strong>t 2.2) .................................................. 25<br />

2.3 Assess<strong>in</strong>g likelihood <strong>of</strong> susta<strong>in</strong>able impact (Control Po<strong>in</strong>t 2.3) .................................................. 26<br />

2.4 Manag<strong>in</strong>g <strong>in</strong>dicators (Control Po<strong>in</strong>t 2.4) ..................................................................................... 26<br />

2.5 Projections (Control Po<strong>in</strong>ts 2.5) .................................................................................................. 26<br />

2.6 Key Resources ............................................................................................................................. 27<br />

3 <strong>Measur<strong>in</strong>g</strong> Changes <strong>in</strong> Indicators ........................................................................................................ 27<br />

3.1 Collect<strong>in</strong>g basel<strong>in</strong>e <strong>in</strong>formation (Control Po<strong>in</strong>t 3.1) ................................................................... 28<br />

3.2 Good research practices (Control Po<strong>in</strong>t 3.2) ............................................................................... 29<br />

3.3 Qualitative <strong>in</strong>formation (Control Po<strong>in</strong>t 3.3) ................................................................................ 32<br />

3.4 Key resources .............................................................................................................................. 33<br />

4 Estimat<strong>in</strong>g Attributable Changes ........................................................................................................ 33<br />

4.1 A clear and appropriate system for attribution is <strong>in</strong> place (Control Po<strong>in</strong>t 4.1) .......................... 34<br />

4.2 Key resources .............................................................................................................................. 36<br />

5 Captur<strong>in</strong>g Wider Changes <strong>in</strong> the System or Market (Control Po<strong>in</strong>ts 5.1) ........................................... 36<br />

6 Track<strong>in</strong>g Programme Costs (Control Po<strong>in</strong>ts 6.1 – 6.2) ........................................................................ 37<br />

7 Report<strong>in</strong>g Results (Control Po<strong>in</strong>ts 7.1 – 7.5) ....................................................................................... 37<br />

8 Manag<strong>in</strong>g the System for Results Measurement ................................................................................ 38<br />

8.1 Establish<strong>in</strong>g a clear system for results measurement (Control Po<strong>in</strong>ts 8.1) ................................ 38<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

1


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

8.2 Human and f<strong>in</strong>ancial resources (Control Po<strong>in</strong>t 8.2) .................................................................... 39<br />

8.3 Key resources .............................................................................................................................. 40<br />

Annex 1: Key Def<strong>in</strong>itions ............................................................................................................................. 41<br />

Annex 2: Acronyms and Abbreviations .............................................................................................. 44<br />

Annex 3: Overall Resource List ............................................................................................................... 45<br />

List <strong>of</strong> Figures<br />

FIGURE 1: MAPPING COMPLEX CONFLICTS…………………………………………………………………………………………………………………. 7<br />

FIGURE 2: THE PSD-PEACEBUILDING CONTINUUM ............................................................................................................ 8<br />

FIGURE 3: MAPPING POSSIBLE NEGATIVE UNINTENTIONAL RESULTS .................................................................................... 13<br />

FIGURE 4: INDICATOR SELECTION TOOL…………………………………………………………………………………………………………………… 26<br />

FIGURE 5: FACTORS REPORTEDLY CONTRIBUTING TO IMPROVED FOOD SECURITY .................................................................... 35<br />

List <strong>of</strong> Tables<br />

TABLE 1: SIMPLIFIED EXAMPLE RESULTS CHAINS OF HOW PSD CAN CONTRIBUTE TO PEACEBUILDING…..…………………………………...15<br />

TABLE 2: CHALLENGING COMMON CONFLICT-RELATED ASSUMPTIONS ................................................................................... 17<br />

TABLE 3: DEFINING CONFLICT-SENSITIVE PSD INDICATORS .................................................................................................. 21<br />

TABLE 4: DEFINING INDICATORS TO TEST CONFLICT-RELEVANT RISKS ..................................................................................... 22<br />

TABLE 5: SAMPLE INDICATORS FOR MEASURING POSITIVE EFFECTS ON CONFLICT ..................................................................... 24<br />

Acknowledgements<br />

The authors wish to thank the many <strong>in</strong>terviewees who provided helpful guidance as we conducted our<br />

research. We are also grateful to the members <strong>of</strong> the Donor Committee for Enterprise <strong>Development</strong><br />

(<strong>DCED</strong>) for their support throughout, as well as Jim Tanburn and Mel<strong>in</strong>a He<strong>in</strong>rich from the <strong>DCED</strong><br />

Secretariat. We dedicate these guidel<strong>in</strong>es to the practitioners who are mak<strong>in</strong>g the effort to design and<br />

implement rigorous results measurement systems <strong>in</strong> conflict affected environments and dissem<strong>in</strong>ate<br />

their results to the rest <strong>of</strong> the field. These efforts are <strong>in</strong>valuable to advanc<strong>in</strong>g our knowledge <strong>in</strong> an area<br />

where much rema<strong>in</strong>s to be learned.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

2


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

I. Introduction<br />

This paper provides guidance for measur<strong>in</strong>g results <strong>in</strong> conflict affected environments (CAEs) us<strong>in</strong>g the<br />

Donor Committee for Enterprise <strong>Development</strong> (<strong>DCED</strong>) Standard for Results Measurement 1 . The <strong>DCED</strong><br />

Standard is a framework for measur<strong>in</strong>g and manag<strong>in</strong>g the results <strong>of</strong> private sector development<br />

programmes. The guidel<strong>in</strong>es follow the <strong>DCED</strong>’s def<strong>in</strong>ition <strong>of</strong> CAEs: “countries or regions where there is a<br />

high risk <strong>of</strong> violent conflict break<strong>in</strong>g out; that are <strong>in</strong> the midst <strong>of</strong> violent conflict; or have recently<br />

emerged from it, <strong>in</strong>clud<strong>in</strong>g countries classified as ‘post-conflict’”. 2 Def<strong>in</strong>itions for this and other<br />

commonly-used terms are provided <strong>in</strong> Annex 1, while acronyms and abbreviations are provided <strong>in</strong><br />

Annex 2.<br />

S<strong>in</strong>ce 2008, the <strong>DCED</strong> Standard has been implemented by numerous private sector development (PSD)<br />

projects across Africa and Asia. Feedback from practitioners <strong>in</strong>dicates that it helps monitor progress<br />

towards objectives and manage <strong>in</strong>terventions. In complex, rapidly chang<strong>in</strong>g environments it is<br />

particularly important for programmes to monitor, learn from and adapt their approach. 3 The emphasis<br />

<strong>of</strong> the <strong>DCED</strong> Standard on us<strong>in</strong>g monitor<strong>in</strong>g data to improve implementation is thus particularly valuable<br />

<strong>in</strong> CAEs, support<strong>in</strong>g the management <strong>of</strong> complex programmes and reduc<strong>in</strong>g the risk <strong>of</strong> caus<strong>in</strong>g harm<br />

through <strong>in</strong>appropriate <strong>in</strong>terventions. Nevertheless, the challenges <strong>of</strong> security, data availability, and<br />

staff<strong>in</strong>g have limited the application <strong>of</strong> the <strong>DCED</strong> Standard <strong>in</strong> CAEs to date.<br />

Donors are <strong>in</strong>creas<strong>in</strong>gly dedicat<strong>in</strong>g resources towards private sector development <strong>in</strong> CAEs, recognis<strong>in</strong>g<br />

the poor performance <strong>of</strong> CAEs towards meet<strong>in</strong>g development objectives such as the Millennium<br />

<strong>Development</strong> Goals. With greater fund<strong>in</strong>g, donors and implementers are be<strong>in</strong>g called upon to develop<br />

rigorous approaches for results measurement and report<strong>in</strong>g, <strong>in</strong> order to demonstrate impact and<br />

accountability. The <strong>DCED</strong> Standard is widely recognized as represent<strong>in</strong>g good practice <strong>in</strong> results<br />

measurement for PSD programm<strong>in</strong>g. Provid<strong>in</strong>g guidance <strong>in</strong> how to apply the Standard <strong>in</strong> CAEs can<br />

support practitioners to measure, manage and demonstrate results.<br />

II. How to Use these Guidel<strong>in</strong>es<br />

These guidel<strong>in</strong>es are written for donors, project implementers, auditors and consultants who are<br />

apply<strong>in</strong>g the <strong>DCED</strong> Standard <strong>in</strong> CAEs or consider<strong>in</strong>g its use. They are structured around the eight<br />

elements <strong>of</strong> the <strong>DCED</strong> Standard. For each component they note the challenges that affect application <strong>in</strong><br />

CAEs, provide guidance on mitigation strategies and refer to additional resources. Resources listed <strong>in</strong><br />

blue text l<strong>in</strong>k to a downloadable resource, and a full list <strong>of</strong> guidance documents are provided <strong>in</strong> Annex 3.<br />

Guidance is provided on how to structure a Standard-compliant results measurement system that, at a<br />

m<strong>in</strong>imum, avoids exacerbat<strong>in</strong>g conflict. For practitioners wish<strong>in</strong>g to use private sector development<br />

1 For more <strong>in</strong>formation about the <strong>DCED</strong> Standard, please refer to:<br />

http://www.enterprise-development.org/page/measur<strong>in</strong>g-and-report<strong>in</strong>g-results.<br />

2 The Donor Committee for Enterprise <strong>Development</strong>, <strong>Private</strong> sector development <strong>in</strong> conflict affected<br />

environments: Key Resources for Practitioners, 2010. The CAE def<strong>in</strong>ition is understood to also encompass the<br />

concept <strong>of</strong> fragility.<br />

3 Hummelbrunner, Richard and Jones, Harry, A guide for plann<strong>in</strong>g and strategy development <strong>in</strong> the face <strong>of</strong><br />

complexity, 2013, Overseas <strong>Development</strong> Institute.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

3


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

programmes as a means to build peace and promote stability, these guidel<strong>in</strong>es outl<strong>in</strong>e an approach for<br />

reflect<strong>in</strong>g this aim with<strong>in</strong> your results measurement system.<br />

In light <strong>of</strong> the great diversity among CAEs, readers should use these guidel<strong>in</strong>es as a resource <strong>in</strong><br />

develop<strong>in</strong>g and implement<strong>in</strong>g their monitor<strong>in</strong>g system, rather than a prescriptive set <strong>of</strong> rules. Many <strong>of</strong><br />

the recommendations are also relevant to PSD programm<strong>in</strong>g <strong>in</strong> humanitarian crises and other<br />

challeng<strong>in</strong>g contexts.<br />

These guidel<strong>in</strong>es assume that users have a basic knowledge <strong>of</strong> the <strong>DCED</strong> Standard and surround<strong>in</strong>g<br />

guidance, as described <strong>in</strong> the next section.<br />

III.<br />

If You Are New to the <strong>DCED</strong> Standard<br />

The <strong>DCED</strong> is a forum for bilateral donors and multilateral agencies seek<strong>in</strong>g to improve the effectiveness<br />

<strong>of</strong> their PSD programm<strong>in</strong>g.<br />

The <strong>DCED</strong> has published a Standard for <strong>Measur<strong>in</strong>g</strong> Results <strong>in</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong>, which<br />

provides a framework for programme managers to monitor their progress and report credible results. It<br />

can be implemented by the programme team and audited by an <strong>in</strong>dependent assessor to verify the<br />

credibility <strong>of</strong> the system. There are eight elements to the Standard, listed below. Follow the hyperl<strong>in</strong>ked<br />

bold text to download further guidance on each:<br />

1. Articulat<strong>in</strong>g Results Cha<strong>in</strong>s. Results cha<strong>in</strong>s visually represents how project activities will create<br />

outputs, outcomes and impact, show<strong>in</strong>g the causal l<strong>in</strong>ks and relationships between them. The<br />

results cha<strong>in</strong>s clearly demonstrate what the project is do<strong>in</strong>g and what changes are expected.<br />

2. Def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators <strong>of</strong> change. An <strong>in</strong>dicator specifies what you will measure <strong>in</strong> order to see whether<br />

change has occurred. By def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators, you clarify exactly how you will monitor your<br />

<strong>in</strong>terventions. Indicators are specified for each expected change outl<strong>in</strong>ed <strong>in</strong> results cha<strong>in</strong>s.<br />

3. <strong>Measur<strong>in</strong>g</strong> changes <strong>in</strong> <strong>in</strong>dicators. Once the <strong>in</strong>dicators have been def<strong>in</strong>ed, they are regularly<br />

monitored to see what has changed to help projects manage accord<strong>in</strong>gly.<br />

4. Estimat<strong>in</strong>g attributable changes. Once a change is observed, you need to estimate what can be<br />

attributed to your project. For example, an <strong>in</strong>crease <strong>in</strong> jobs may be due to your project – or because<br />

<strong>of</strong> the wider economic environment. Estimat<strong>in</strong>g attributable changes helps a project identify which<br />

<strong>in</strong>terventions are work<strong>in</strong>g and which are not.<br />

5. Captur<strong>in</strong>g wider changes <strong>in</strong> the system or market. Many PSD programmes aim to affect entire<br />

market systems. Monitor<strong>in</strong>g these changes gives a project a fuller picture <strong>of</strong> its impact, helps<br />

identify what is work<strong>in</strong>g and revise implementation strategies to maximize results.<br />

6. Track<strong>in</strong>g programme costs. In order to assess the success <strong>of</strong> the project it is necessary to know how<br />

much was spent to achieve the results.<br />

7. Report<strong>in</strong>g results. F<strong>in</strong>d<strong>in</strong>gs are communicated clearly to funders, local stakeholders, and to the<br />

wider development community where possible.<br />

8. Manag<strong>in</strong>g the system for results measurement. For a monitor<strong>in</strong>g and results measurement system<br />

to be effective, it must be adequately resourced and <strong>in</strong>tegrated <strong>in</strong>to all aspects <strong>of</strong> project<br />

management, <strong>in</strong>form<strong>in</strong>g the implementation and guid<strong>in</strong>g the strategy.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

4


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

The Standard divides each <strong>of</strong> these broad categories <strong>in</strong>to control po<strong>in</strong>ts and compliance criteria. The<br />

control po<strong>in</strong>ts summarise the exact requirements, while the compliance criteria describe how an auditor<br />

will assess the programme.<br />

Readers who are unfamiliar with the Standard can visit the <strong>DCED</strong>’s page on measur<strong>in</strong>g results to learn<br />

more about the <strong>DCED</strong> Standard. You can also download the Standard itself, or read the comprehensive<br />

implementation guidel<strong>in</strong>es downloadable through the above hyperl<strong>in</strong>ks or from the website here. It is<br />

advisable to review those documents before read<strong>in</strong>g these guidel<strong>in</strong>es.<br />

IV.<br />

Key Pr<strong>in</strong>ciples to Implement<strong>in</strong>g the <strong>DCED</strong> Standard <strong>in</strong> CAEs<br />

Three pr<strong>in</strong>ciples underlie the application <strong>of</strong> the <strong>DCED</strong> Standard <strong>in</strong> conflict affected areas: simplicity,<br />

flexibility and sensitivity.<br />

Simplicity. Personnel <strong>in</strong> CAEs are <strong>of</strong>ten overworked and under-resourced, with little tra<strong>in</strong><strong>in</strong>g <strong>in</strong> results<br />

measurement. High staff turnover may reduce familiarity with the project and context. An overly<br />

complex results measurement system will not be successfully implemented, wast<strong>in</strong>g project resources<br />

and reduc<strong>in</strong>g the will<strong>in</strong>gness <strong>of</strong> staff to use it.<br />

These guidel<strong>in</strong>es highlight techniques for simplify<strong>in</strong>g the results measurement system that enable it to<br />

be implemented even <strong>in</strong> challeng<strong>in</strong>g CAEs.<br />

Flexibility. CAEs are unpredictable, complicated, and fast-chang<strong>in</strong>g. An <strong>in</strong>flexible results measurement<br />

system may monitor irrelevant <strong>in</strong>dicators, fail to capture the positive impact <strong>of</strong> the project, and put staff<br />

and project clients at risk by ignor<strong>in</strong>g negative consequences.<br />

Consequently, your results measurement system should be flexible. Be aware that a results cha<strong>in</strong> can<br />

never capture the full complexity <strong>of</strong> the situation, and may become rapidly outdated if not regularly<br />

reviewed. Be alert for positive and negative changes not captured by your <strong>in</strong>dicators, and be prepared<br />

to modify the monitor<strong>in</strong>g system accord<strong>in</strong>gly. Aga<strong>in</strong>, these guidel<strong>in</strong>es suggest ways to manage this<br />

complexity while implement<strong>in</strong>g the <strong>DCED</strong> Standard.<br />

Sensitivity. Inappropriate <strong>in</strong>terventions <strong>in</strong> CAEs can endanger staff, partners and project clients while<br />

worsen<strong>in</strong>g the conflict. PSD projects must be particularly aware <strong>of</strong> potential negative impacts because<br />

conflicts are frequently driven by economic factors, which PSD <strong>in</strong>terventions can either re<strong>in</strong>force or<br />

reduce. For example, <strong>in</strong>creas<strong>in</strong>g the level <strong>of</strong> competition among firms can exacerbate conflict if one side<br />

perceives that their bus<strong>in</strong>esses are be<strong>in</strong>g displaced.<br />

Accord<strong>in</strong>gly, all projects work<strong>in</strong>g <strong>in</strong> CAEs should ’Do No Harm’. This means that they must understand<br />

the context <strong>in</strong> which they operate, understand the <strong>in</strong>teraction between their activities and the context,<br />

and act upon this understand<strong>in</strong>g to avoid negative impacts. 4 The results measurement system should<br />

thus monitor potential negative effects <strong>of</strong> the project on the conflict. Some programmes may wish to go<br />

4 Conflict Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peacebuild<strong>in</strong>g Resource Pack, 2004,<br />

3.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

5


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

further and explicitly aim to promote peace and stability through PSD. In this case, they may also<br />

monitor positive effects <strong>of</strong> the project on the conflict, although this poses greater challenges. Moreover,<br />

the monitor<strong>in</strong>g system must itself take the conflict <strong>in</strong>to account. Ask<strong>in</strong>g politically sensitive questions <strong>in</strong><br />

<strong>in</strong>terviews or shar<strong>in</strong>g identifiable <strong>in</strong>formation can put staff and beneficiaries at risk. These guidel<strong>in</strong>es<br />

highlight ways to <strong>in</strong>tegrate conflict sensitivity <strong>in</strong>to the results measurement system.<br />

Note that the follow<strong>in</strong>g sections will follow the sequence <strong>of</strong> the <strong>DCED</strong> Standard elements 1-8 and<br />

associated control po<strong>in</strong>ts.<br />

1 Articulat<strong>in</strong>g Results Cha<strong>in</strong>s<br />

Results cha<strong>in</strong>s visually articulate the activities, outputs, outcomes and impact <strong>of</strong> your project and the<br />

l<strong>in</strong>kages between them. This demonstrates what the project is do<strong>in</strong>g and what changes are expected as<br />

a result. If you have not worked with results cha<strong>in</strong>s before, you can download the <strong>DCED</strong> Results Cha<strong>in</strong><br />

guide here for a complete explanation.<br />

Results cha<strong>in</strong>s are important tools for results measurement <strong>in</strong> all projects, <strong>in</strong>clud<strong>in</strong>g those implemented<br />

<strong>in</strong> a CAE. The process <strong>of</strong> design<strong>in</strong>g results cha<strong>in</strong>s clarifies the logic <strong>of</strong> your project, forc<strong>in</strong>g you to be<br />

explicit about what changes you expect to see and how they relate to the desired outcomes. This allows<br />

your assumptions to be closely exam<strong>in</strong>ed and questioned, <strong>in</strong> order to check that the activities will<br />

plausibly lead to the planned results. Results cha<strong>in</strong>s also create the basis for results measurement. By<br />

articulat<strong>in</strong>g the logic <strong>of</strong> your <strong>in</strong>tervention, the results cha<strong>in</strong>s outl<strong>in</strong>e expected results. This allows you to<br />

regularly check whether your project is on track.<br />

Results Cha<strong>in</strong>s and Theories <strong>of</strong> Change<br />

The <strong>DCED</strong> Standard requires the use <strong>of</strong> results cha<strong>in</strong>s for each project <strong>in</strong>tervention, whereas the<br />

peacebuild<strong>in</strong>g community more frequently refers to theories <strong>of</strong> change. A theory <strong>of</strong> change is an<br />

approach to the design, monitor<strong>in</strong>g and evaluation <strong>of</strong> social programmes. It requires a clear articulation<br />

<strong>of</strong> the activities <strong>of</strong> the project and how it expects them to <strong>in</strong>teract with the context and key actors to<br />

lead to long-term change. This is referred to as the ‘logic’ <strong>of</strong> the project. It frequently <strong>in</strong>cludes a<br />

consideration <strong>of</strong> the context and key actors. Theories <strong>of</strong> change are sometimes depicted visually.<br />

Results cha<strong>in</strong>s are one way to visualise a theory <strong>of</strong> change that represents the series <strong>of</strong> specific changes<br />

that l<strong>in</strong>k project activities to impacts us<strong>in</strong>g a series <strong>of</strong> boxes and arrows. Whether you speak <strong>of</strong> results<br />

cha<strong>in</strong>s or theories <strong>of</strong> change, the key to their successful use is to cont<strong>in</strong>ually test the underly<strong>in</strong>g<br />

assumptions and use that to improve the project.<br />

There are several challenges to develop<strong>in</strong>g effective results cha<strong>in</strong>s <strong>in</strong> CAEs. These <strong>in</strong>clude:<br />

Knowledge gaps about market systems <strong>in</strong> CAEs. A good results cha<strong>in</strong> requires an <strong>in</strong>-depth<br />

knowledge <strong>of</strong> targeted market systems. For example, a PSD programme might seek to facilitate<br />

improvements <strong>in</strong> the quality <strong>of</strong> farmers’ produce, <strong>in</strong> order to <strong>in</strong>crease the prices that the farmer<br />

receives. This relies on the assumption that market pric<strong>in</strong>g is sensitive to product quality, which will<br />

not always be true. However, <strong>in</strong> a CAE it is <strong>of</strong>ten challeng<strong>in</strong>g to assess market systems, for a number<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

6


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

<strong>of</strong> reasons. There may be few people who understand market development <strong>in</strong> the local area, the<br />

market environment may have been rapidly<br />

chang<strong>in</strong>g as a result <strong>of</strong> the conflict, and it may be<br />

hard to gather new <strong>in</strong>formation due to a lack <strong>of</strong><br />

secondary <strong>in</strong>formation or formal <strong>in</strong>stitutions.<br />

Complexity. CAEs have multiple actors and<br />

numerous causes <strong>of</strong> conflict. This can result <strong>in</strong> a<br />

complex <strong>in</strong>teraction between the project and the<br />

context. Expected results have multiple causes, and<br />

project activities can have multiple effects – not all<br />

<strong>of</strong> which are l<strong>in</strong>ear or predictable. However, the<br />

results cha<strong>in</strong> design must balance complexity and<br />

usability. If it is overly simple, it will not capture the<br />

project logic. If it is overly complex, such as <strong>in</strong> the<br />

Figure 1: Mapp<strong>in</strong>g Complex Conflicts<br />

This excerpt from a diagram <strong>of</strong> the Afghan context<br />

shows how complex any attempt to fully represent a<br />

conflict can be.<br />

adjacent figure, 5 it is difficult to read and monitor aga<strong>in</strong>st. While this is true <strong>in</strong> every context, the<br />

presence <strong>of</strong> conflict <strong>in</strong>troduces additional complications that make this balanc<strong>in</strong>g act particularly<br />

challeng<strong>in</strong>g.<br />

Fast pace <strong>of</strong> change. Rapid change can render a results cha<strong>in</strong> out <strong>of</strong> date. For example,<br />

<strong>in</strong>tensification <strong>of</strong> a conflict may make market actors unwill<strong>in</strong>g to cont<strong>in</strong>ue participat<strong>in</strong>g <strong>in</strong> an<br />

<strong>in</strong>tervention and thus <strong>in</strong>validate that <strong>in</strong>tervention’s results cha<strong>in</strong>.<br />

Knowledge gaps about the relationship between PSD and peacebuild<strong>in</strong>g. While there are theories<br />

about the relationship between PSD <strong>in</strong>terventions and peacebuild<strong>in</strong>g, there is limited evidence on<br />

the detailed causal relationships, or the contexts <strong>in</strong> which those relationships hold. 6 For example, a<br />

recent systematic review by the Overseas <strong>Development</strong> Institute found that, out <strong>of</strong> 9,558 articles<br />

reviewed, only one provided any evidence regard<strong>in</strong>g whether or not employment creation<br />

promoted stability. 7<br />

1.1 Incorporat<strong>in</strong>g conflict <strong>in</strong>to results cha<strong>in</strong>s (Control Po<strong>in</strong>t 1.1-1.2)<br />

Develop<strong>in</strong>g and ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>g results cha<strong>in</strong>s requires staff to cont<strong>in</strong>ually exam<strong>in</strong>e their own assumptions<br />

and validate the l<strong>in</strong>kages between levels <strong>of</strong> their results cha<strong>in</strong>s. This is important <strong>in</strong> a CAE, where l<strong>in</strong>ks<br />

that are valid <strong>in</strong> a stable context may not hold. For example:<br />

Assumption: adequate trust exists for economic relationships to be built. Economic engagement<br />

between members <strong>of</strong> different groups requires trust. The United States Agency for International<br />

<strong>Development</strong> (USAID) found that ‘[b]ecause conflict breaks down trust between people, some forms<br />

<strong>of</strong> support and collaboration will only be possible when basic elements <strong>of</strong> trust are restored.’ 8<br />

5 Diagram cited <strong>in</strong> Banyan Global, Enterprise <strong>Development</strong>, Poverty and Conflict, 2010.<br />

6 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict & Economics: Lessons<br />

Learned on <strong>Measur<strong>in</strong>g</strong> Impact, 2011, 6.<br />

7 Holmes et al, What is the evidence on the impact <strong>of</strong> employment creation on stability and poverty reduction <strong>in</strong><br />

fragile states: A systematic review, 2013, v.<br />

8 USAID, Conflict and Livelihoods, 2005, 11.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

7


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Assumption: the private sector is will<strong>in</strong>g to engage. <strong>Private</strong> firms <strong>in</strong> CAEs may be less will<strong>in</strong>g to<br />

participate <strong>in</strong> PSD <strong>in</strong>itiatives than <strong>in</strong> other environments. Inter-group prejudices, security and other<br />

concerns can prevent the establishment <strong>of</strong> partnerships. The non-governmental organization (NGO)<br />

International Alert, for <strong>in</strong>stance, found that the private sector was “not overly enthusiastic” about<br />

provid<strong>in</strong>g employment opportunities to ex-combatants who had received vocational tra<strong>in</strong><strong>in</strong>g but<br />

who represented a potential security risk. 9 A long history <strong>of</strong> conflict may also have significantly<br />

deteriorated human capital as well; ex-combatants may not have basic skills required for<br />

employment (e.g. literacy).<br />

The results cha<strong>in</strong> design will depend on the project’s approach to address<strong>in</strong>g conflict. There are three<br />

broad approaches that PSD projects may use <strong>in</strong> CAEs:<br />

Conflict Bl<strong>in</strong>d. Projects do not <strong>in</strong>corporate the conflict <strong>in</strong>to their background research, design,<br />

implementation, or monitor<strong>in</strong>g.<br />

Do No Harm. Projects consciously look for and seek to avoid or mitigate negative impacts result<strong>in</strong>g<br />

from their activities. 10 Do<strong>in</strong>g No Harm requires the use <strong>of</strong> a conflict sensitive 11 approach.<br />

Peacebuild<strong>in</strong>g. Projects explicitly design their PSD <strong>in</strong>terventions to reduce conflict and build peace.<br />

A peacebuild<strong>in</strong>g project will aim to identify and address the underly<strong>in</strong>g cause(s) <strong>of</strong> conflict.<br />

Figure 2: The PSD-Peacebuild<strong>in</strong>g Cont<strong>in</strong>uum 12<br />

Focus on Conflict Resolution and Peacebuild<strong>in</strong>g<br />

<strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> Programm<strong>in</strong>g<br />

Conflict Bl<strong>in</strong>d Do No Harm Peacebuild<strong>in</strong>g<br />

Does not take conflict <strong>in</strong>to<br />

account <strong>in</strong> design,<br />

implementation, monitor<strong>in</strong>g,<br />

or evaluation<br />

Risks fail<strong>in</strong>g to meet<br />

objectives, and potentially<br />

caus<strong>in</strong>g harm<br />

Understands the conflict and<br />

conflict drivers<br />

Seeks to m<strong>in</strong>imize potential<br />

project harm by monitor<strong>in</strong>g<br />

un<strong>in</strong>tended adverse effects<br />

<strong>of</strong> project on conflict<br />

Results cha<strong>in</strong> articulates<br />

expected positive impacts <strong>of</strong><br />

project on reduc<strong>in</strong>g conflict<br />

Incorporates <strong>in</strong>dicators to<br />

test positive impacts <strong>of</strong><br />

project on conflict<br />

9 Godnick, William and Diana Kle<strong>in</strong>, The challenges <strong>of</strong> support<strong>in</strong>g ‘alternative’ economic opportunities for<br />

peacebuild<strong>in</strong>g – Perspectives from Colombia, 2009, 25.<br />

10 Interpeace, What is Peacebuild<strong>in</strong>g, 2010.<br />

11 A conflict sensitive project understands the context <strong>in</strong> which it works, comprehends the <strong>in</strong>teraction between its<br />

actions and the context and acts upon this understand<strong>in</strong>g <strong>in</strong> order to avoid negative impacts and maximise positive<br />

ones.<br />

12 Prepared by the authors, adapted from Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for<br />

projects <strong>in</strong> conflict affected countries, Phase 1: Formative research and key issue diagnosis, undated, 28.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

8


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

A “conflict bl<strong>in</strong>d” project may not achieve its objectives, due to unanticipated conflict-related factors, or<br />

<strong>in</strong>advertently worsen the conflict. The next two sections therefore explore how the project logic can<br />

<strong>in</strong>corporate Do No Harm and peacebuild<strong>in</strong>g pr<strong>in</strong>ciples.<br />

1.1.1 Do No Harm<br />

At a m<strong>in</strong>imum, projects <strong>in</strong> CAEs should aim<br />

to “Do No Harm”. They must be built on a<br />

comprehensive conflict analysis, and<br />

consider potential negative <strong>in</strong>teractions<br />

between the conflict and the context.<br />

1.1.1.1 Conflict Analysis<br />

“At the beg<strong>in</strong>n<strong>in</strong>g <strong>of</strong> the project, I was not<br />

conv<strong>in</strong>ced by peace and conflict analysis. I<br />

thought that I don’t have time for it. But<br />

I’ve discovered that, if I make the time, we<br />

avoid a lot <strong>of</strong> trouble. A good conflict<br />

analysis is vital to success.” – Liane Hryca,<br />

GIZ Afghanistan 13<br />

To be effective <strong>in</strong> a CAE, the project logic<br />

needs to be based on an understand<strong>in</strong>g <strong>of</strong><br />

the conflict. This should be done through a<br />

conflict analysis, which is a systematic<br />

study <strong>of</strong> the pr<strong>of</strong>ile, causes, actors, and dynamics <strong>of</strong> conflict. 14<br />

In particular, the conflict analysis should<br />

identify the drivers (i.e. underly<strong>in</strong>g causes) <strong>of</strong> conflict. For a selection <strong>of</strong> conflict analysis guidel<strong>in</strong>es, see<br />

the key resources section below.<br />

Key Questions for Conflict Analysis 15<br />

Conflict pr<strong>of</strong>ile<br />

What is the political, economic, and socio-cultural context<br />

What are the emergent political, economic, ecological and social issues<br />

What specific conflict prone/affected areas can be situated with<strong>in</strong> this context<br />

Is there a history <strong>of</strong> conflict<br />

Conflict causes<br />

What are structural causes <strong>of</strong> conflict<br />

What issues can be considered as proximate causes <strong>of</strong> conflict<br />

Impact <strong>of</strong> EU engagement <strong>in</strong> Afar Region, Ethiopia<br />

Afar is an <strong>in</strong>secure region <strong>in</strong> the East <strong>of</strong> Ethiopia, where<br />

regional <strong>in</strong>security, weak regional government, loss <strong>of</strong><br />

access to livelihood resources, and marg<strong>in</strong>alization <strong>of</strong><br />

m<strong>in</strong>ority groups, <strong>in</strong> particular pastoralist communities,<br />

all risk creat<strong>in</strong>g physical conflict.<br />

The EU provided approximately two billion Euros<br />

between 1976 and 1994 <strong>in</strong> aid to Ethiopia. In the Afar<br />

region, this <strong>in</strong>cluded large-scale irrigation projects on<br />

more than 40,000 ha <strong>of</strong> land.<br />

A report by Saferworld concluded that these projects<br />

have excluded pastoralists from plann<strong>in</strong>g and<br />

implementation, depriv<strong>in</strong>g them <strong>of</strong> graz<strong>in</strong>g land and<br />

<strong>in</strong>creas<strong>in</strong>g the risk <strong>of</strong> conflict between pastoralists,<br />

agriculturalists and the state.<br />

What triggers can contribute to the outbreak/further escalation <strong>of</strong> conflict<br />

13 Interview with Liane Hryca (GIZ Afghanistan), September 28, 2012.<br />

14 Conflict Sensitivity Consortium, How To Guide to Conflict Sensitivity, 2012.<br />

15 Conflict Sensitivity Consortium, How To Guide to Conflict Sensitivity, 2012.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

9


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Conflict actors<br />

Who are the ma<strong>in</strong> actors<br />

What are their ma<strong>in</strong> <strong>in</strong>terest, goals, positions, capacities, and relationships<br />

What <strong>in</strong>stitutional capacities for peace can be identified<br />

What actors can be identified as (potential) spoilers Why What are their <strong>in</strong>centives<br />

Conflict dynamics<br />

What are the current conflict trends<br />

What are w<strong>in</strong>dows <strong>of</strong> opportunity<br />

What scenarios can be developed from the analysis <strong>of</strong> the conflict pr<strong>of</strong>ile, causes and actors<br />

A conflict analysis for a PSD programme should pay particular attention to how economic factors divide<br />

and connect parties <strong>in</strong> a conflict, and exam<strong>in</strong>e the role played by economic actors. For example, <strong>in</strong> longterm<br />

conflicts warlords <strong>of</strong>ten effectively become bus<strong>in</strong>ess people, controll<strong>in</strong>g natural resources and<br />

provid<strong>in</strong>g employment. The conflict analysis should exam<strong>in</strong>e who benefits and loses economically from<br />

the conflict, and how external <strong>in</strong>tervention is likely to affect and be affected by the conflict. It should<br />

also exam<strong>in</strong>e the potential for the private sector to be a participant <strong>in</strong> the peace process, by <strong>of</strong>fer<strong>in</strong>g<br />

jobs to ex-combatants or support<strong>in</strong>g negotiations between different parties. 16<br />

The conflict analysis can be <strong>in</strong>corporated <strong>in</strong>to other analyses that PSD projects typically carry out to<br />

<strong>in</strong>form their design, such as market systems analysis or value cha<strong>in</strong> analysis. The <strong>DCED</strong> publication PSD<br />

<strong>in</strong> CAEs: Key Resources for Practitioners 17 describes seven tools that practitioners can use to design<br />

conflict-sensitive PSD programmes <strong>in</strong> CAEs. For example, GIZ <strong>in</strong> Sierra Leone has <strong>in</strong>corporated conflict<br />

sensitivity <strong>in</strong>to their selection <strong>of</strong> value cha<strong>in</strong>s. As well as evaluat<strong>in</strong>g economic potential, participatory<br />

workshops <strong>in</strong> each district explore the extent to which the value cha<strong>in</strong>s br<strong>in</strong>g different groups together,<br />

and the risks <strong>of</strong> foster<strong>in</strong>g <strong>in</strong>creased competition on conflict. 18<br />

F<strong>in</strong>d<strong>in</strong>gs from the conflict analysis will allow you to m<strong>in</strong>imise negative impacts on conflict and maximize<br />

positive ones by <strong>in</strong>corporat<strong>in</strong>g them <strong>in</strong>to the project design and results measurement system. This<br />

establishes an accurate programme logic, enables the development <strong>of</strong> conflict-sensitive <strong>in</strong>dicators, and<br />

ensures that changes <strong>in</strong> the context can be monitored and reflected <strong>in</strong> project design and the<br />

monitor<strong>in</strong>g system. This may be easier to achieve if staff conduct and analyse the conflict analysis,<br />

rather than outsourc<strong>in</strong>g it. 19 Ideally the <strong>in</strong>itial conflict analysis should be conducted prior to or dur<strong>in</strong>g the<br />

programme design phase, so that its f<strong>in</strong>d<strong>in</strong>gs can <strong>in</strong>form the programme. Critically, the conflict analysis<br />

must be regularly updated to detect any changes <strong>in</strong> the context and adjust <strong>in</strong>terventions that may<br />

exacerbate the conflict.<br />

16 One useful checklist is GTZ, Checklist: Economic Aspects <strong>of</strong> a Conflict<br />

17 <strong>DCED</strong>, <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflicted Affected Areas: Key Resources for Practitioners, 2010.<br />

18 Interview with Beatrice Tsch<strong>in</strong>kel (GIZ), September 17, 2012<br />

19 Conflict Sensitivity Consortium, How To Guide to Conflict Sensitivity, 2012, 5.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

10


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

1.1.1.2 Consider<strong>in</strong>g a project’s potential negative effects through results cha<strong>in</strong>s<br />

A PSD project can create or exacerbate conflict <strong>in</strong> various ways, <strong>in</strong>clud<strong>in</strong>g:<br />

Real or perceived bias <strong>in</strong> the distribution <strong>of</strong> project resources, support or employment<br />

Resource transfers to parties to the conflict<br />

Enabl<strong>in</strong>g a diversion <strong>of</strong> other resources to conflict (e.g. warr<strong>in</strong>g parties are freed from their<br />

obligation to support civilians)<br />

Contribut<strong>in</strong>g to <strong>in</strong>flation<br />

Chang<strong>in</strong>g exist<strong>in</strong>g power structures, for example by <strong>in</strong>creas<strong>in</strong>g competition <strong>in</strong> a market.<br />

Reduc<strong>in</strong>g the capacity <strong>of</strong> local structures. For example, recruit<strong>in</strong>g local government staff or activists<br />

(either formally or through payment <strong>of</strong> per diems for project work) or sett<strong>in</strong>g up parallel bodies. 20<br />

Consequently, it is essential for each project to consider its possible negative consequences. One wellestablished<br />

methodology for do<strong>in</strong>g this is the Do No Harm (DNH) approach. This is “an effort to identify<br />

the ways <strong>in</strong> which <strong>in</strong>ternational development assistance given <strong>in</strong> conflict sett<strong>in</strong>gs may be provided so<br />

that, rather than exacerbat<strong>in</strong>g and worsen<strong>in</strong>g the conflict, it helps local people disengage from fight<strong>in</strong>g<br />

and develop systems for settl<strong>in</strong>g the problems which prompt conflict with<strong>in</strong> their societies.” The<br />

methodology has been widely applied and has a large amount <strong>of</strong> guidance available. 21<br />

<strong>Private</strong> sector development programmes work<strong>in</strong>g <strong>in</strong> CAEs should adopt the DNH approach or a similar<br />

methodology to help them th<strong>in</strong>k about their effects on conflict. Several ways to <strong>in</strong>corporate the DNH<br />

approach <strong>in</strong> the creation <strong>of</strong> results cha<strong>in</strong>s are highlighted below.<br />

Case Study: Apply<strong>in</strong>g a Do No Harm Approach to PSD Interventions <strong>in</strong> Somaliland<br />

The USAID-funded Partnership for Economic Growth project, implemented by DAI <strong>in</strong> Somaliland, is<br />

focused on promot<strong>in</strong>g private sector development while us<strong>in</strong>g a Do No Harm approach. The project uses<br />

a multi-pronged approach to m<strong>in</strong>imize potential negative impacts <strong>of</strong> the project on clan-based tensions:<br />

1. Dur<strong>in</strong>g any market research and project design stages, the team conducts an ‘analysis <strong>of</strong> local political<br />

economy’. This maps potential scenarios <strong>in</strong> which project resources will <strong>in</strong>flame clan tensions, such as by<br />

<strong>in</strong>terven<strong>in</strong>g <strong>in</strong> value cha<strong>in</strong>s that are dom<strong>in</strong>ated by specific clans. This analysis allows the project to test<br />

that its <strong>in</strong>terventions will properly support competitiveness strategies while ensur<strong>in</strong>g clan balance.<br />

2. The project’s Communications and Community Outreach Officer and the security focal po<strong>in</strong>t<br />

cont<strong>in</strong>ually <strong>in</strong>form the project team <strong>of</strong> any possible land conflicts or related tensions <strong>in</strong> and around<br />

project <strong>in</strong>tervention areas and work closely with local government agencies and civil society<br />

stakeholders to ensure that all project activities are supported by neighbour<strong>in</strong>g communities.<br />

20 Anderson and Olson. Confront<strong>in</strong>g War: Critical Lessons for Peace Practitioners, 2003, 22-28.<br />

21 See http://www.cda<strong>in</strong>c.com/cdawww/project_pr<strong>of</strong>ile.phppid=DNH&pname=Do%20No%20Harm<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

11


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

3. The project will not directly support a bus<strong>in</strong>ess <strong>in</strong> a sector where there is a great deal <strong>of</strong> competition<br />

already (and therefore tensions may be exacerbated) or where donor support risks creat<strong>in</strong>g unfair<br />

advantages for bus<strong>in</strong>ess owners <strong>in</strong> one clan over those <strong>in</strong> another.<br />

4. Dur<strong>in</strong>g the project’s bus<strong>in</strong>ess match<strong>in</strong>g grants competition, all short-listed f<strong>in</strong>alists are requested to<br />

submit a “Bus<strong>in</strong>ess Risks and Assumptions Report” to be reviewed by the project team. With<strong>in</strong> this<br />

report, the grantee is required to lay out their “Do No Harm approach” dur<strong>in</strong>g the bus<strong>in</strong>ess expansion<br />

activity that the project will be support<strong>in</strong>g. Through site visits and follow up with key <strong>in</strong>dustry<br />

stakeholders, this report is verified before bus<strong>in</strong>ess match<strong>in</strong>g grants activities beg<strong>in</strong>.<br />

5. The project uses formal (e.g. through government) and <strong>in</strong>formal (e.g. between non-governmental<br />

organizations and local communities) communication approaches to expla<strong>in</strong> the importance <strong>of</strong> <strong>in</strong>direct<br />

benefits (e.g. new employment creation) aris<strong>in</strong>g from PSD <strong>in</strong>terventions and to therefore reduce<br />

pressure for direct project <strong>in</strong>terventions with all households.<br />

Identify conflict-related risks at each level <strong>of</strong> results cha<strong>in</strong>s<br />

Develop<strong>in</strong>g results cha<strong>in</strong>s can help th<strong>in</strong>k through the conflict-related risks at all levels, from activity to<br />

impact. While research<strong>in</strong>g these guidel<strong>in</strong>es, the authors found that practitioners tended to effectively<br />

consider short term risks caused by activities, such as conflict over distribution <strong>of</strong> resources. However, it<br />

is rarer for practitioners to consider how the outputs and outcomes <strong>of</strong> their projects might affect<br />

conflict. This step is essential <strong>in</strong> order to fully understand and manage the risks <strong>of</strong> the project.<br />

For example, risks created by the activities <strong>of</strong> a programme might <strong>in</strong>clude:<br />

Step <strong>in</strong> Results Cha<strong>in</strong><br />

Organisation provides entrepeneurship tra<strong>in</strong><strong>in</strong>g to<br />

participants<br />

Organisation identifies participants <strong>in</strong> tra<strong>in</strong><strong>in</strong>g<br />

session<br />

Conflict Related Risks<br />

Non-participants feel that the selection<br />

process was unfair.<br />

Potential participants compete over who<br />

gets selected.<br />

Local conflict parties require space <strong>in</strong><br />

tra<strong>in</strong><strong>in</strong>g for their own supporters.<br />

At the outcome level, risks might <strong>in</strong>clude:<br />

Step <strong>in</strong> Results Cha<strong>in</strong><br />

Participants start up their own<br />

bus<strong>in</strong>esses<br />

Participants <strong>in</strong>crease knowledge on how<br />

to start a bus<strong>in</strong>ess<br />

Conflict Related Risks<br />

New bus<strong>in</strong>esses conflict with exist<strong>in</strong>g<br />

bus<strong>in</strong>esses <strong>in</strong> same sector.<br />

New bus<strong>in</strong>esses pay taxes to local warlord,<br />

provid<strong>in</strong>g resources that fuel conflict.<br />

Participants can’t start bus<strong>in</strong>esses due to<br />

other constra<strong>in</strong>ts, and feel frustrated due<br />

to raised expectations.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

12


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Visually depict serious risks alongside results cha<strong>in</strong>s<br />

Serious risks can be represented more fully by creat<strong>in</strong>g a secondary strand <strong>in</strong> a results cha<strong>in</strong>. This will<br />

allow you to either adjust your activities to address the risk or – if you th<strong>in</strong>k the potential risk is small<br />

and the impact low – to assign <strong>in</strong>dicators to ensure that these risks are closely monitored. For example,<br />

the follow<strong>in</strong>g figure presents one example from East Timor. The project developed an agreement with<br />

the village community on the use <strong>of</strong> land and water, enabl<strong>in</strong>g young people to cultivate land and<br />

<strong>in</strong>tegrate <strong>in</strong>to communal life. This was expected to reduce the potential for conflict. However, the<br />

agreements would reduce the overall land available for each villager, potentially creat<strong>in</strong>g a new source<br />

<strong>of</strong> conflict, as identified on the left side <strong>of</strong> the results cha<strong>in</strong>. 22<br />

Figure 3:<br />

Mapp<strong>in</strong>g Possible Negative Un<strong>in</strong>tentional Results<br />

As serious risks are identified, it may be necessary to adjust exist<strong>in</strong>g activities or develop additional<br />

activities to mitigate the sources <strong>of</strong> conflict. These adjustments and their results should be <strong>in</strong>corporated<br />

dur<strong>in</strong>g subsequent updates to your results cha<strong>in</strong>s.<br />

Be aware that results cha<strong>in</strong>s <strong>of</strong>ten cannot capture all <strong>of</strong> the complex <strong>in</strong>teractions between conflict and<br />

your project. Be alert for unanticipated negative effects, and responsive to feedback from the field staff<br />

and community.<br />

22 Lange, Ralf et al.: Results and Results-based Monitor<strong>in</strong>g <strong>of</strong> the Contributions made by Vocational Education &<br />

Tra<strong>in</strong><strong>in</strong>g and the Promotion <strong>of</strong> Employment to Social (Re-)Integration <strong>in</strong> (Post-)Conflict Situations. Draft version –<br />

Unpublished, 2011, BMZ/GIZ, 11.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

13


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

1.1.1.3 Consider<strong>in</strong>g potential negative effects <strong>of</strong> conflict through results cha<strong>in</strong>s<br />

In addition to exam<strong>in</strong><strong>in</strong>g a project’s positive or negative impacts on conflict, it is also important to<br />

consider the conflict’s potential impact on project activities. These may <strong>in</strong>clude:<br />

Direct attacks on or threats to staff members or the project, limit<strong>in</strong>g their potential for engagement<br />

Beneficiaries chose not to participate <strong>in</strong> the project, because they do not see it as neutral or do not<br />

consider it safe to do so<br />

Inability to access project location, h<strong>in</strong>der<strong>in</strong>g implementation and monitor<strong>in</strong>g<br />

Project assets stolen by warr<strong>in</strong>g groups<br />

Initial project design rendered irrelevant by <strong>in</strong>tensification <strong>of</strong> conflict (e.g. project beneficiaries are<br />

forced to move)<br />

Such risks to project performance can be <strong>in</strong>corporated with<strong>in</strong> the risks and assumptions section <strong>of</strong> your<br />

project design documents. Once identified, they should <strong>in</strong>form the choice <strong>of</strong> project <strong>in</strong>terventions and<br />

strategies to mitigate their impacts.<br />

1.1.2 Incorporat<strong>in</strong>g Peacebuild<strong>in</strong>g <strong>in</strong>to PSD<br />

Conflict is <strong>of</strong>ten driven by economic factors 23 , and so PSD may play a critical role <strong>in</strong> creat<strong>in</strong>g the<br />

conditions for a stable society. It can address the causes or escalat<strong>in</strong>g factors <strong>of</strong> conflict, such as<br />

unemployment or poverty. It can also support peacebuild<strong>in</strong>g and reconciliation processes, for example<br />

by support<strong>in</strong>g the bus<strong>in</strong>ess community to negotiate between different parties to the conflict.<br />

Projects may consequently wish to build their programme logic around contribut<strong>in</strong>g to peacebuild<strong>in</strong>g.<br />

However, not all projects should try to capture their impact on conflict, as this requires adequate staff<br />

capacity, time, and fund<strong>in</strong>g. Few PSD projects have thus far been able to do so effectively. With<br />

sufficient resources, however, it is a very valuable project outcome to monitor.<br />

If projects did wish to capture their impact on stability, they should first outl<strong>in</strong>e their theory <strong>of</strong> change,<br />

or results cha<strong>in</strong>, expla<strong>in</strong><strong>in</strong>g how they expect to have a positive impact on the conflict. This should be<br />

rooted <strong>in</strong> the conflict analysis, and aim to address the identified drivers <strong>of</strong> conflict. To <strong>in</strong>form the<br />

development <strong>of</strong> peacebuild<strong>in</strong>g results cha<strong>in</strong>s, consider the follow<strong>in</strong>g list, which presents several <strong>of</strong> the<br />

most recognized ways <strong>in</strong> which PSD can contribute to peacebuild<strong>in</strong>g. 24 In practice, results cha<strong>in</strong>s are<br />

likely to be far more complex than these examples, with multiple levels and branches. For one example,<br />

see the <strong>DCED</strong> case study <strong>of</strong> GIZ <strong>in</strong> Sierra Leone.<br />

23 Love, Roy, Economic Drivers <strong>of</strong> Conflict and Cooperation <strong>in</strong> the Horn <strong>of</strong> Africa: A Regional Perspective and<br />

Overview, 2009.<br />

24 Drawn and adapted from The Donor Committee for Enterprise <strong>Development</strong>, <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong><br />

Conflict-Affected Environments – A Framework <strong>of</strong> Peacebuild<strong>in</strong>g Impacts, 2010; Rachel Goldwyn, Livelihoods and<br />

Peacebuild<strong>in</strong>g – what are current approaches to contribut<strong>in</strong>g to peacebuild<strong>in</strong>g through livelihoods <strong>in</strong>terventions<br />

(unpublished); Church, Cheyanne and Mark M. Roberts, Design<strong>in</strong>g for Results: Integrat<strong>in</strong>g M&E <strong>in</strong> Conflict<br />

Transformation Programmes, 2006; Mercy Corps. Evaluation and Assessment <strong>of</strong> Poverty and Conflict<br />

Interventions. Conflict & Economics: Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Undated.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

14


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Table 1: Simplified example results cha<strong>in</strong>s <strong>of</strong> how PSD can contribute to peacebuild<strong>in</strong>g<br />

Simplified Results Cha<strong>in</strong><br />

Higher levels<br />

<strong>of</strong> trust<br />

Reduced conflict<br />

Reduced levels <strong>of</strong> conflict<br />

Fewer young people out <strong>of</strong> work<br />

Improved economic opportunities<br />

for young people<br />

PSD <strong>in</strong>tervention<br />

Reduced conflict over resources<br />

Greater economic opportunities that<br />

do not rely on over-exploited<br />

resources<br />

PSD <strong>in</strong>tervention<br />

Increased economic<br />

<strong>in</strong>centives for peace<br />

Greater economic <strong>in</strong>teraction<br />

among parties to conflict<br />

PSD Intervention<br />

Explanation<br />

PSD <strong>in</strong>terventions foster greater economic <strong>in</strong>teraction among<br />

parties to a conflict, which creates economic <strong>in</strong>centives for<br />

peace and reduces the desire for conflict.<br />

Mercy Corps’ Build<strong>in</strong>g Bridges to Peace program <strong>in</strong> Uganda<br />

supported jo<strong>in</strong>t economic <strong>in</strong>itiatives such as farm<strong>in</strong>g and<br />

marketplace rehabilitation between groups that had<br />

previously clashed <strong>in</strong> the Northeast Karamoja region.<br />

Compared with communities that did not receive project<br />

activities, the targeted villages reported had better economic<br />

opportunities, reduced perceptions <strong>of</strong> <strong>in</strong>security and stronger<br />

<strong>in</strong>teraction between ethnic groups. 25<br />

PSD <strong>in</strong>terventions create <strong>in</strong>come and employment<br />

opportunities that reduce the <strong>in</strong>centives to engage <strong>in</strong> conflict.<br />

For <strong>in</strong>dividuals with no <strong>in</strong>come or jobs, conflict may be the<br />

only way to support themselves or their families. Even if they<br />

do not fight themselves, they may participate <strong>in</strong> war<br />

economies, such as by grow<strong>in</strong>g drugs. Employment<br />

opportunities <strong>in</strong> peaceful vocations reduce this <strong>in</strong>centive to go<br />

to war. This theory is <strong>of</strong>ten applied by ex-combatant<br />

re<strong>in</strong>tegration programmes. For example, a re<strong>in</strong>tegration<br />

programme <strong>in</strong> Colombia reviewed by International Alert 26<br />

<strong>of</strong>fered ex-combatants grants for livelihoods projects,<br />

employment, or further education. They reported that over<br />

14,300 ex-combatants had found employment.<br />

PSD <strong>in</strong>terventions <strong>in</strong>crease economic opportunities that do<br />

not rely on over-exploited resources, thereby reduc<strong>in</strong>g a key<br />

driver <strong>of</strong> conflict.<br />

Conflicts are <strong>of</strong>ten driven by <strong>in</strong>adequate access to resources<br />

that are critical to livelihoods (e.g. water, graz<strong>in</strong>g land).<br />

Negotiated agreements over the use <strong>of</strong> such resources are<br />

most susta<strong>in</strong>able where groups have other economic<br />

opportunities. In Somalia, Mercy Corps supported the<br />

negotiation <strong>of</strong> eight agreements to limit competition for scarce<br />

resources <strong>in</strong>clud<strong>in</strong>g trees while simultaneously support<strong>in</strong>g<br />

alternative economic opportunities such as nursery<br />

management. These alternative livelihood opportunities can<br />

reduce the prevalence <strong>of</strong> economic activities that put a stra<strong>in</strong><br />

on over-exploited resources (e.g. charcoal production). 27<br />

25 Mercy Corps, Peacebuild<strong>in</strong>g through Economic <strong>Development</strong> Approach, 2011, 2.<br />

26 Godnick and Kle<strong>in</strong>, The challenges <strong>of</strong> support<strong>in</strong>g ‘Alternative’ economic opportunities for peacebuild<strong>in</strong>g –<br />

perspectives from Colombia, 2009, 24<br />

27 Mercy Corps, Peacebuild<strong>in</strong>g through Economic <strong>Development</strong> Approach, 2011, 3.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

15


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Simplified Results Cha<strong>in</strong><br />

Reduced<br />

diversion <strong>of</strong><br />

resources to<br />

parties to the<br />

conflict<br />

Reduced conflict<br />

PSD Intervention<br />

Reduced conflict<br />

Reduced tension between<br />

different groups<br />

Reduced <strong>in</strong>equality<br />

Increased flow <strong>of</strong><br />

resources to<br />

government (e.g.<br />

through tax)<br />

<strong>Private</strong> sector improves governance<br />

Explanation<br />

PSD <strong>in</strong>terventions improve the governance <strong>of</strong> the private<br />

sector, which <strong>in</strong> turn puts pressure on the state to improve its<br />

own governance and support conflict reduction. As a major<br />

societal actor, the decisions and behaviour <strong>of</strong> the private<br />

sector has a major role <strong>in</strong> <strong>in</strong>fluenc<strong>in</strong>g government behaviour<br />

and actions. Initiatives such as the Extractive Industries<br />

Transparency Initiative encourage private firms to behave<br />

responsibly and transparently <strong>in</strong> their relations with the state,<br />

report<strong>in</strong>g what payments are made and to whom. This can<br />

reduce conflict both directly, by deny<strong>in</strong>g combatants the funds<br />

needed to fuel conflict, and <strong>in</strong>directly, as the private sector<br />

<strong>in</strong>creases use <strong>of</strong> formal state channels and processes – rather<br />

than circumvent<strong>in</strong>g these processes through pay<strong>in</strong>g bribes. A<br />

stronger and better resourced government is <strong>in</strong> a stronger<br />

position to reduce conflict. However, <strong>in</strong> many contexts the<br />

government is also a cause <strong>of</strong> context – so this assumption (as<br />

with the others listed here) would have to be validated<br />

through a conflict analysis.<br />

PSD <strong>in</strong>terventions reduce <strong>in</strong>equality between parties to a<br />

conflict, thus reduc<strong>in</strong>g resentment and the likelihood <strong>of</strong><br />

conflict. Where conflict is driven by resentment and anger<br />

over <strong>in</strong>equitable distribution <strong>of</strong> resources, PSD <strong>in</strong>terventions<br />

that support economic improvements by marg<strong>in</strong>alized<br />

populations may reduce these tensions and thus support<br />

peace.<br />

PSD <strong>in</strong>tervention<br />

Each <strong>of</strong> the above theories is based on assumptions about the underly<strong>in</strong>g causes <strong>of</strong> conflict, which will<br />

not hold true <strong>in</strong> every situation. The logic <strong>of</strong> the results cha<strong>in</strong> should be <strong>in</strong>formed by a rigorous conflict<br />

analysis that identifies the drivers <strong>of</strong> conflict and analyses how the project can best address them.<br />

Results cha<strong>in</strong>s will make these assumptions explicit. PSD <strong>in</strong>terventions do not automatically have a<br />

positive impact on a conflict, 28 so your monitor<strong>in</strong>g system should research and test them. The table<br />

below presents common parts <strong>of</strong> peacebuild<strong>in</strong>g results cha<strong>in</strong>s and exam<strong>in</strong>es the assumptions underly<strong>in</strong>g<br />

them.<br />

28 Gündüz, Canan and Diane Kle<strong>in</strong>, Conflict Sensitive Approaches To Value Cha<strong>in</strong> <strong>Development</strong>, 2008, 21.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

16


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Table 1:<br />

Challeng<strong>in</strong>g Common Conflict-Related Assumptions<br />

Step <strong>in</strong> Results Cha<strong>in</strong><br />

Increased trust<br />

between conflict<strong>in</strong>g<br />

groups<br />

Increased economic<br />

<strong>in</strong>teraction betwen<br />

conflict<strong>in</strong>g groups<br />

Reduced <strong>in</strong>centive to<br />

engage <strong>in</strong> conflict<br />

Increased<br />

employment/<strong>in</strong>come<br />

Reduced<br />

conflict<br />

Reduced<br />

<strong>in</strong>equality<br />

Assumptions<br />

Economic <strong>in</strong>teraction will build trust. Economic <strong>in</strong>teractions may<br />

reduce rather than <strong>in</strong>crease levels <strong>of</strong> trust where they <strong>in</strong>crease<br />

competition. For example, a jo<strong>in</strong>t marketplace could have a negative<br />

impact if different groups are compet<strong>in</strong>g for sales. It is likely that some<br />

k<strong>in</strong>ds <strong>of</strong> economic relationships contribute more strongly than others<br />

to build<strong>in</strong>g trust. Mercy Corps observed that “‘Deep’ economic<br />

<strong>in</strong>teractions (such as participation <strong>in</strong> economic associations or bus<strong>in</strong>ess<br />

partnerships) may build stronger relationships between adversarial<br />

groups and provide a stronger <strong>in</strong>centive for peace than ‘th<strong>in</strong>’ economic<br />

<strong>in</strong>teractions (such as trad<strong>in</strong>g at a local market).” 29<br />

Conflict is primarily driven by desire for additional resources. Conflict<br />

frequently has multiple causes, <strong>in</strong>clud<strong>in</strong>g ideology, discrim<strong>in</strong>ation, and<br />

<strong>in</strong>equality. The causes will vary from country to country. The World<br />

Bank found that, across seven countries, 40% <strong>of</strong> rebels jo<strong>in</strong> because <strong>of</strong><br />

unemployment or idleness – while only 13% jo<strong>in</strong> because <strong>of</strong> belief <strong>in</strong><br />

the cause. 30 But <strong>in</strong> Mali and the West Bank the trend was reversed,<br />

with 46% cit<strong>in</strong>g belief <strong>in</strong> the cause, and about 20% cit<strong>in</strong>g<br />

unemployment or idleness. Consequently, this theory <strong>of</strong> change would<br />

likely be less appropriate <strong>in</strong> the latter two areas.<br />

Lower levels <strong>of</strong> <strong>in</strong>equality reduce the risk <strong>of</strong> conflict. While overall<br />

<strong>in</strong>equality <strong>in</strong> the economy is not associated with a higher risk <strong>of</strong><br />

conflict, case study evidence suggests that horizontal <strong>in</strong>equality<br />

between regions or ethnic groups is. 31 Horizontal <strong>in</strong>equality <strong>of</strong>ten<br />

correlates with overall <strong>in</strong>equality, but not always; a society might have<br />

high levels <strong>of</strong> <strong>in</strong>equality between ethnic groups, but low overall levels<br />

<strong>of</strong> <strong>in</strong>equality. If a PSD project seeks to reduce conflict by reduc<strong>in</strong>g<br />

<strong>in</strong>equality, it should clarify <strong>in</strong> its results cha<strong>in</strong> which groups it expects<br />

to work with, and how <strong>in</strong>equality between these groups causes<br />

conflict.<br />

Project learn<strong>in</strong>g and adaptation is a critical to develop<strong>in</strong>g effective results cha<strong>in</strong>s <strong>in</strong> CAEs. The follow<strong>in</strong>g<br />

case study <strong>of</strong> the International Rescue Committee’s project <strong>in</strong> Liberia emphasizes the importance <strong>of</strong><br />

collect<strong>in</strong>g <strong>in</strong>formation to test a project’s assumptions about the drivers <strong>of</strong> conflict. 32<br />

29 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions Conflict & Economics: Lessons<br />

Learned on <strong>Measur<strong>in</strong>g</strong> Impact, undated, 5.<br />

30 World Bank, World <strong>Development</strong> Report 2011: Conflict, Security and <strong>Development</strong>, 2011, 80.<br />

31 Humphreys, Macartan, Economies and Violent Conflict, 2002, 3-4.<br />

32 Fearon, James et al, Evaluat<strong>in</strong>g Community-Driven Reconstruction: Lessons from post-conflict Liberia.<br />

<strong>Development</strong> Outreach, 2009.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

17


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Case Study: Test<strong>in</strong>g Assumptions <strong>in</strong> Liberia<br />

International Rescue Committee’s Community Driven Reconstruction project sought to <strong>in</strong>crease social<br />

cohesion with<strong>in</strong> villages <strong>in</strong> Liberia, on the assumption that this had been destroyed by many years <strong>of</strong><br />

civil war. The logic <strong>of</strong> one <strong>of</strong> their project’s <strong>in</strong>terventions – provid<strong>in</strong>g fund<strong>in</strong>g for committees to spend<br />

on community projects – is illustrated by the simplified results cha<strong>in</strong> below. The results cha<strong>in</strong> shows<br />

the project assumed their <strong>in</strong>terventions would lead to <strong>in</strong>creased cohesion, improved attitudes to<br />

governance, and improved participation <strong>of</strong> m<strong>in</strong>ority groups. However, although the <strong>in</strong>tervention did<br />

<strong>in</strong>crease levels <strong>of</strong> cohesion, democratic attitudes and social acceptance, the basel<strong>in</strong>e discovered that<br />

these were high with<strong>in</strong> treatment and control communities alike before the start <strong>of</strong> the project. This<br />

challenged the underly<strong>in</strong>g assumption <strong>of</strong> the project that years <strong>of</strong> civil war had significantly reduced<br />

the social cohesion with<strong>in</strong> village. The evaluation found it plausible that “the really important divisions<br />

exist between communities rather than with<strong>in</strong> them”. This f<strong>in</strong>d<strong>in</strong>g underscores the importance <strong>of</strong><br />

test<strong>in</strong>g important assumptions prior to and dur<strong>in</strong>g implementation and reflect<strong>in</strong>g this learn<strong>in</strong>g <strong>in</strong><br />

project <strong>in</strong>terventions <strong>in</strong> order to ensure that they have their desired effects.<br />

1.2 Manag<strong>in</strong>g the results cha<strong>in</strong> (Control Po<strong>in</strong>ts 1.3 – 1.4)<br />

In order to successfully manage results cha<strong>in</strong>s <strong>in</strong> a CAE:<br />

Keep it simple. With high staff turnover and limited resources, there is typically limited capacity to<br />

use and manage results cha<strong>in</strong>s. Consider reduc<strong>in</strong>g the number <strong>of</strong> steps to show only key changes<br />

that are expected, or splitt<strong>in</strong>g a bigger <strong>in</strong>tervention <strong>in</strong>to smaller, more manageable results cha<strong>in</strong>s.<br />

Adapt the complexity to the capacity <strong>of</strong> the team.<br />

Regularly review and revise (as needed). There is a risk that the use <strong>of</strong> results cha<strong>in</strong>s can lead<br />

project staff to adopt an <strong>in</strong>flexible, l<strong>in</strong>ear approach. This is <strong>in</strong>appropriate <strong>in</strong> CAEs, where project staff<br />

need the flexibility to respond to unpredictable situations. The results cha<strong>in</strong>s should be reviewed<br />

regularly (at least every six months), and adjusted when circumstances change or the orig<strong>in</strong>al results<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

18


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

cha<strong>in</strong>s are <strong>in</strong>validated. 33 They should not be considered a prescriptive guide to programme activities,<br />

but a way to explore and challenge the assumptions <strong>of</strong> the programme.<br />

Consult all stakeholders. This is particularly important <strong>in</strong> a conflict situation, where you need<br />

acceptance from multiple groups. Stakeholder consultations can themselves have a positive impact<br />

upon the conflict by creat<strong>in</strong>g opportunities for shar<strong>in</strong>g and jo<strong>in</strong>t ownership. Consider whether<br />

groups <strong>in</strong> conflict should be consulted jo<strong>in</strong>tly or separately. However, be aware that groups may be<br />

biased towards particular positions, and consultations should be carefully planned so as not to<br />

favour any particular party. GTZ notes that “particularly <strong>in</strong> direct post-conflict situations,<br />

organisations and <strong>in</strong>stitutions with direct l<strong>in</strong>ks to conflict parties are <strong>of</strong>ten the most visible but not<br />

necessarily the most eligible partners.” 34 This is an extremely difficult issue for many projects <strong>in</strong><br />

CAEs, and it is essential to take the time to understand the different group dynamics and consult<br />

widely.<br />

Support project staff to understand the results cha<strong>in</strong>s and use it to drive their decisions. In a CAE<br />

various factors can m<strong>in</strong>imize staff understand<strong>in</strong>g <strong>of</strong> results cha<strong>in</strong>s, <strong>in</strong>clud<strong>in</strong>g high staff turnover,<br />

<strong>in</strong>creased use <strong>of</strong> partners, and low staff capacity. Consider mak<strong>in</strong>g it part <strong>of</strong> the <strong>in</strong>duction for new<br />

staff, and hold<strong>in</strong>g regular tra<strong>in</strong><strong>in</strong>g on it. For more strategies to address human resources issues, see<br />

section 8 on manag<strong>in</strong>g the results measurement system.<br />

1.3 Key resources<br />

1.3.1 Conflict Analysis Guidel<strong>in</strong>es<br />

Conflict Sensitivity Consortium. Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian<br />

Assistance and Peace Build<strong>in</strong>g Resource Pack, Chapter Two. Very comprehensive conflict sensitivity<br />

guidel<strong>in</strong>es. Chapter two describes conflict analysis <strong>in</strong> detail and summarises 15 different conflict analysis<br />

tools from major donors. For each they discuss the primary purpose, summarise the ma<strong>in</strong> steps, and<br />

give examples <strong>of</strong> lessons learned and current applications. If you are select<strong>in</strong>g a conflict analysis<br />

framework, this is an <strong>in</strong>valuable resource.<br />

Collaborative for <strong>Development</strong> Action, Inc. The Do No Harm Handbook: The Framework for Analyz<strong>in</strong>g<br />

the Impact <strong>of</strong> Assistance on Conflict. This guide provides a step-by-step guide to assess<strong>in</strong>g the potential<br />

<strong>of</strong> a project to cause harm <strong>in</strong> its selected context.<br />

GIZ. Peace and Conflict Assessment (PCA): A Methodological Framework for the Conflict- and Peace-<br />

Oriented Alignment <strong>of</strong> <strong>Development</strong> Programmes. Exam<strong>in</strong>es the application <strong>of</strong> peace and conflict<br />

assessments at multiple stages <strong>in</strong> a project, <strong>in</strong>clud<strong>in</strong>g impact monitor<strong>in</strong>g.<br />

USAID. Conflict Assessment Framework. Version 2.0. Provides a framework for diagnos<strong>in</strong>g a conflict<br />

and generat<strong>in</strong>g responses. Application guidel<strong>in</strong>es can be found here.<br />

1.3.2 Results Cha<strong>in</strong>s and Theories <strong>of</strong> Change<br />

Becker, Sab<strong>in</strong>e. Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes. Reviews<br />

many <strong>of</strong> the key relationships between PSD and peacebuild<strong>in</strong>g.<br />

33 Vogel, Isabel, Review <strong>of</strong> the use <strong>of</strong> ‘Theory Of Change’ In <strong>in</strong>ternational development’, 2012, 27.<br />

34 Becker, Sab<strong>in</strong>e, Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes, 2006, GTZ<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

19


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

CARE International UK. Peacebuild<strong>in</strong>g with Impact: Def<strong>in</strong><strong>in</strong>g Theories <strong>of</strong> Change. Offers guidance <strong>in</strong><br />

develop<strong>in</strong>g the logic <strong>of</strong> how project <strong>in</strong>terventions will impact upon peacebuild<strong>in</strong>g.<br />

Davis, Peter. Corporations, Global Governance and Post-Conflict Reconstruction. Synthesizes the<br />

impacts corporations may have on conflict and reconstruction, draw<strong>in</strong>g from Azerbaijan, Bosnia and<br />

Rwanda.<br />

Humphreys, Macartan. Economics and Violent Conflict. Reviews the evidence for l<strong>in</strong>kages between<br />

economics and conflict.<br />

Mercy Corps. Peacebuild<strong>in</strong>g through Economic <strong>Development</strong> Approach. Outl<strong>in</strong>es five theories <strong>of</strong><br />

change that Mercy Corps is implement<strong>in</strong>g and test<strong>in</strong>g through its work and provides examples <strong>of</strong> each.<br />

Mierke, Axel. Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes. Analyses the<br />

potential contribution <strong>of</strong> PSD programm<strong>in</strong>g to peacebuild<strong>in</strong>g.<br />

Pottebaum, David. Relationships between Conflict, Poverty, Inequality, and Economic Growth.<br />

Reviews the evidence for relationships between conflict, poverty, <strong>in</strong>equality, and economic growth.<br />

Sen, Nabanita. A Guide to Mak<strong>in</strong>g Results Cha<strong>in</strong>s. Provides the key steps required to create a results<br />

cha<strong>in</strong>.<br />

USAID. Conflict-Sensitive Approaches to Value Cha<strong>in</strong> <strong>Development</strong>. Outl<strong>in</strong>es how to <strong>in</strong>corporate a<br />

conflict lens when conduct<strong>in</strong>g value cha<strong>in</strong>s analysis. The f<strong>in</strong>d<strong>in</strong>gs can <strong>in</strong>form the logic <strong>of</strong> a project’s<br />

results cha<strong>in</strong>s.<br />

Vogel, Isabel. Review <strong>of</strong> the uses <strong>of</strong> ‘Theory <strong>of</strong> Change’ <strong>in</strong> International <strong>Development</strong>. Exam<strong>in</strong>es how<br />

theories <strong>of</strong> change are used <strong>in</strong> the field, their different applications and best practices.<br />

1.3.3 Conflict Sensitivity<br />

Conflict Sensitivity Consortium. Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian<br />

Assistance and Peace Build<strong>in</strong>g Resource Pack. Provides guidance on how to <strong>in</strong>corporate a conflict<br />

sensitive approach throughout the project cycle, <strong>in</strong>clud<strong>in</strong>g monitor<strong>in</strong>g and evaluation.<br />

Conflict Sensitivity Consortium. A How To Guide to Conflict Sensitivity. A more concise and updated<br />

version <strong>of</strong> the above document.<br />

2 Def<strong>in</strong><strong>in</strong>g Indicators <strong>of</strong> Change<br />

A results measurement system should only track what you are attempt<strong>in</strong>g to <strong>in</strong>fluence through your<br />

work, or factors that will have a strong <strong>in</strong>fluence on your work. This ensures that you do not spend time<br />

and money measur<strong>in</strong>g irrelevant factors, which is particularly important <strong>in</strong> a CAE, due to the common<br />

constra<strong>in</strong>ts <strong>of</strong> money and staff time. The <strong>DCED</strong> Standard requires <strong>in</strong>dicators to be l<strong>in</strong>ked to the results<br />

cha<strong>in</strong>, <strong>in</strong> order to ensure that they are relevant to the project.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

20


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

A pr<strong>in</strong>ciple challenge to def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators <strong>in</strong> CAEs is that relevant conflict-relevant <strong>in</strong>dicators are still<br />

be<strong>in</strong>g understood globally – particularly at the impact level – and their applicability across contexts is<br />

unclear. 35<br />

2.1 Def<strong>in</strong><strong>in</strong>g <strong>in</strong>dicators for results cha<strong>in</strong>s (Control Po<strong>in</strong>t 2.1)<br />

The <strong>DCED</strong> Standard requires the track<strong>in</strong>g <strong>of</strong> each change <strong>in</strong> a results cha<strong>in</strong>. Projects need to carefully<br />

assess the changes they must measure to assess the theory <strong>of</strong> change, recogniz<strong>in</strong>g that capacity and<br />

resources are typically very limited <strong>in</strong> a CAE.<br />

PSD projects <strong>in</strong> CAEs should disaggregate key <strong>in</strong>dicators by parties to the conflict, and <strong>in</strong>corporate<br />

additional <strong>in</strong>dicators that would not be necessary <strong>in</strong> stable environments. As <strong>in</strong>troduced <strong>in</strong> Section 1, all<br />

projects should set <strong>in</strong>dicators to monitor the negative effects <strong>of</strong> their work, and ensure that they are not<br />

<strong>in</strong>advertently caus<strong>in</strong>g harm. Projects with a peacebuild<strong>in</strong>g objective may also wish to set <strong>in</strong>dicators to<br />

exam<strong>in</strong>e the positive impact <strong>of</strong> their work.<br />

2.1.1 Disaggregate key changes by parties to the conflict<br />

Projects should disaggregate key <strong>in</strong>dicators by parties to the conflict, <strong>in</strong> order to:<br />

Ensure that the project is not disproportionately benefit<strong>in</strong>g one group, which might create conflict.<br />

Reveal potential positive impacts on conflict, such as <strong>in</strong>creased ethnic diversity among partners’<br />

workforces.<br />

“If you ask someone about their clan <strong>in</strong> Hargeisa (Somaliland), you will be put on the first plane out <strong>of</strong><br />

there.” – Interviewee, SEED Somalia<br />

While disaggregat<strong>in</strong>g <strong>in</strong>dicators by ethnic group can be helpful, it can also add to tensions. For <strong>in</strong>stance,<br />

<strong>in</strong> Somalia it is generally taboo to ask about clan affiliation. Measurement by geographic area may<br />

therefore be more appropriate where this serves as a proxy for ethnicity. 36 The table below shows some<br />

examples <strong>of</strong> disaggregat<strong>in</strong>g PSD <strong>in</strong>dicators:<br />

Table 2:<br />

Def<strong>in</strong><strong>in</strong>g Conflict-Sensitive PSD Indicators<br />

Standard PSD Indicator<br />

Jobs created as a result <strong>of</strong><br />

programme activities<br />

Change <strong>in</strong> SME net <strong>in</strong>come<br />

Disaggregated PSD Indicator<br />

Jobs created as a result <strong>of</strong> programme activities (disaggregated by<br />

relevant party to the conflict / geographic region)<br />

Change <strong>in</strong> SME net <strong>in</strong>come (disaggregated by relevant party to the<br />

conflict / geographic region)<br />

2.1.2 Assess<strong>in</strong>g negative effects <strong>of</strong> the project on conflict<br />

All projects should use their results measurement system to ensure they avoid conflict. There are<br />

various ways <strong>in</strong> which a PSD project can create or exacerbate conflict, as noted above <strong>in</strong> Section 1.<br />

35 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict & Economics: Lessons<br />

Learned on <strong>Measur<strong>in</strong>g</strong> Impact, undated, 6.<br />

36 ECOPA, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict-affected countries, Phase 2: Case studies and<br />

recommendations, August 2011, 41-42.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

21


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Indicators should be tailored to the identified conflict risks and l<strong>in</strong>ked to project activities and expected<br />

outcomes. They must be <strong>in</strong>formed by the conflict assessment, as different conflicts will have different<br />

observable effects. 37 Indicators can act as early warn<strong>in</strong>g signs <strong>of</strong> conflict and enable programme staff to<br />

understand the long-term effects <strong>of</strong> the programme and manage it accord<strong>in</strong>gly.<br />

At the activity level, simple <strong>in</strong>dicators can serve as early warn<strong>in</strong>g systems <strong>of</strong> the conflict. For example,<br />

the SEED programme <strong>in</strong> Somalia used the follow<strong>in</strong>g <strong>in</strong>dicators:<br />

Boycott <strong>of</strong> programme meet<strong>in</strong>gs by <strong>in</strong>vited parties<br />

Un<strong>of</strong>ficial meet<strong>in</strong>gs with a negative agenda convened locally to discuss the programme<br />

Reluctance by state <strong>of</strong>ficials, <strong>in</strong>terest groups, or programme clients to participate <strong>in</strong> programme<br />

<strong>in</strong>terventions<br />

Compla<strong>in</strong>ts from state <strong>of</strong>ficials or community leaders <strong>of</strong> non-<strong>in</strong>volvement/non-recognition<br />

Segregation <strong>of</strong> certa<strong>in</strong> organized <strong>in</strong>terest groups/<strong>in</strong>dividuals <strong>in</strong> community mobilization<br />

processes<br />

Lack <strong>of</strong> participation by stakeholders <strong>in</strong> project <strong>in</strong>terventions<br />

Threats to staff/partners<br />

Expression <strong>of</strong> grievances or discontent around resource distribution<br />

Use <strong>of</strong> political, social or economic <strong>in</strong>fluence to <strong>in</strong>terfere with programme <strong>in</strong>terventions<br />

For more <strong>in</strong>dicator examples and details on their use, see the full case study <strong>of</strong> the SEED programme on<br />

the <strong>DCED</strong> website.<br />

The close monitor<strong>in</strong>g <strong>of</strong> such <strong>in</strong>dicators – as well as vigilant observation by programme staff <strong>of</strong><br />

unexpected or unusual behaviours – can provide early warn<strong>in</strong>g to a project <strong>of</strong> potential dangers.<br />

Nevertheless, recognize that a project can never anticipate all potential sources <strong>of</strong> conflict and your<br />

conflict <strong>in</strong>dicators will therefore never be comprehensive. Rema<strong>in</strong> vigilant for other sources <strong>of</strong> conflict<br />

that are not captured by your <strong>in</strong>dicators.<br />

At the output and outcome level, <strong>in</strong>dicators should be def<strong>in</strong>ed that are relevant to the conflict and the<br />

programme. Examples are given <strong>in</strong> the table below.<br />

Table 3:<br />

Def<strong>in</strong><strong>in</strong>g Indicators to Test Conflict-Relevant Risks<br />

Box from the<br />

Results Cha<strong>in</strong><br />

Jobs created<br />

Risk<br />

Project resources are seen to (or<br />

actually do) favour some groups over<br />

others<br />

Conflict-Sensitive PSD Indicator<br />

Beneficiaries’ perceptions <strong>of</strong> fairness <strong>of</strong> job<br />

distribution.<br />

Percentage <strong>of</strong> <strong>in</strong>dividuals <strong>in</strong> project area who<br />

a) Are aware <strong>of</strong> the beneficiary target<strong>in</strong>g<br />

criteria.<br />

b) Believe that the target<strong>in</strong>g criteria are fair.<br />

37 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack,<br />

2004.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

22


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

c) Believe that the target<strong>in</strong>g criteria were<br />

correctly applied.<br />

SME revenues<br />

<strong>in</strong>crease<br />

Bus<strong>in</strong>ess<br />

membership<br />

organizations<br />

(BMOs)<br />

attract new<br />

members<br />

Significant imbalance <strong>in</strong> <strong>in</strong>come<br />

change between parties to the conflict<br />

Parties to the conflict are excluded<br />

from the BMO<br />

BMO members compete for project<br />

resources (e.g. tra<strong>in</strong><strong>in</strong>g, per diems).<br />

Change <strong>in</strong> SME net <strong>in</strong>come (disaggregated by<br />

relevant party to the conflict / geographic<br />

region)<br />

Participation <strong>in</strong> BMO (disaggregated by<br />

relevant party to the conflict / geographic<br />

region)<br />

Percentage change <strong>in</strong> will<strong>in</strong>gness to<br />

economically <strong>in</strong>teract with members <strong>of</strong><br />

conflict<strong>in</strong>g communities<br />

Reasons cited by non-members <strong>of</strong> BMOs for<br />

not jo<strong>in</strong><strong>in</strong>g.<br />

Number <strong>of</strong> conflicts caused by competition<br />

for project resources.<br />

2.1.3 <strong>Measur<strong>in</strong>g</strong> positive effects <strong>of</strong> the project on the conflict<br />

PSD projects wish<strong>in</strong>g to have a positive impact on conflict should make this explicit <strong>in</strong> their results cha<strong>in</strong>s<br />

and develop related <strong>in</strong>dicators that are <strong>in</strong>formed by the results <strong>of</strong> a conflict assessment and stakeholder<br />

feedback. This ensures that your conflict <strong>in</strong>dicators are closely tied to the logic <strong>of</strong> your project. It helps<br />

to avoid vague, overarch<strong>in</strong>g <strong>in</strong>dicators, while allow<strong>in</strong>g you to trace changes related to your programme<br />

and build a credible story for why you believe you have had a positive impact.<br />

The table below illustrates <strong>in</strong>dicators for three simplified results cha<strong>in</strong>s. 38 Each one depicts a different<br />

strategy for <strong>in</strong>creas<strong>in</strong>g peace and stability based on vary<strong>in</strong>g assumptions about the source <strong>of</strong> conflict.<br />

Different <strong>in</strong>dicators are therefore required to assess the results cha<strong>in</strong>s’ validity. A range <strong>of</strong> <strong>in</strong>dicators are<br />

listed below for the results <strong>in</strong> the black and white boxes.<br />

38 In practice, results cha<strong>in</strong>s are likely to have multiple levels and branches, as does this one.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

23


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Table 4: Sample Indicators for <strong>Measur<strong>in</strong>g</strong> Positive Effects on Conflict 39<br />

Results Cha<strong>in</strong> Result 1 (Middle black box) Result 2 (Middle white box)<br />

Increase <strong>in</strong> peace and<br />

# <strong>of</strong> bus<strong>in</strong>esses jo<strong>in</strong>tly<br />

stability<br />

owned by members <strong>of</strong><br />

conflict<strong>in</strong>g communities<br />

% change <strong>in</strong><br />

Increase <strong>in</strong> trust between<br />

members <strong>of</strong> conflict<strong>in</strong>g<br />

groups<br />

Increase <strong>in</strong> economic<br />

<strong>in</strong>teractions between<br />

members <strong>of</strong> conflict<strong>in</strong>g<br />

groups<br />

Project supports bus<strong>in</strong>ess<br />

partnership and l<strong>in</strong>kage<br />

forums<br />

Increase <strong>in</strong> peace and<br />

stability<br />

Members <strong>of</strong> conflict<strong>in</strong>g<br />

group have economic<br />

<strong>in</strong>terest <strong>in</strong> peace<br />

Increase <strong>in</strong> economic<br />

<strong>in</strong>teractions between<br />

members <strong>of</strong> conflict<strong>in</strong>g<br />

groups<br />

Project supports bus<strong>in</strong>ess<br />

partnership and l<strong>in</strong>kage<br />

forums<br />

level/frequency/type <strong>of</strong><br />

economic <strong>in</strong>teraction<br />

between members <strong>of</strong><br />

conflict<strong>in</strong>g communities<br />

# <strong>of</strong> bus<strong>in</strong>esses jo<strong>in</strong>tly<br />

owned by members <strong>of</strong><br />

conflict<strong>in</strong>g communities<br />

% change <strong>in</strong><br />

level/frequency/type <strong>of</strong><br />

economic <strong>in</strong>teraction<br />

between members <strong>of</strong><br />

conflict<strong>in</strong>g communities<br />

% change <strong>in</strong> negative<br />

stereotypes about members<br />

<strong>of</strong> “oppos<strong>in</strong>g” groups<br />

% change <strong>in</strong> levels <strong>of</strong> trust<br />

between members <strong>of</strong><br />

different groups<br />

% <strong>of</strong> ex-combatants who, at<br />

the end <strong>of</strong> year X, rout<strong>in</strong>ely<br />

identify themselves as<br />

members <strong>of</strong> the larger<br />

community rather than<br />

belong<strong>in</strong>g to one group or<br />

faction<br />

% change <strong>in</strong> perceived cost <strong>of</strong><br />

conflict with members <strong>of</strong><br />

community X<br />

% change <strong>in</strong> the number <strong>of</strong><br />

people who can articulate a<br />

concrete, tangible benefit<br />

from <strong>in</strong>teract<strong>in</strong>g with an<br />

“oppos<strong>in</strong>g” group.<br />

% change <strong>in</strong> perceived<br />

benefits from economic<br />

<strong>in</strong>teraction with members <strong>of</strong><br />

community X<br />

% change <strong>in</strong> # <strong>of</strong> people who<br />

believe that their economic<br />

well-be<strong>in</strong>g and/or future is<br />

dependent on their economic<br />

relationship with the other<br />

group<br />

39 Drawn from Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict &<br />

Economics: Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact, 2011; DFID, <strong>Measur<strong>in</strong>g</strong> and manag<strong>in</strong>g for results <strong>in</strong> fragile and<br />

conflicted affected states, 2012; and <strong>in</strong>terviewees.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

24


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Increase <strong>in</strong> peace and<br />

stability<br />

Participants have more<br />

economic <strong>in</strong>centives to<br />

avoid war<br />

Participants have more jobs,<br />

<strong>in</strong>creased <strong>in</strong>come<br />

Project supports growth <strong>in</strong><br />

dynamic market systems<br />

% change <strong>in</strong> <strong>in</strong>come<br />

# <strong>of</strong> jobs created<br />

% change <strong>in</strong> satisfaction with<br />

livelihoods<br />

% <strong>of</strong> demobilised excombatants<br />

ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>g an<br />

<strong>in</strong>dependent livelihood after<br />

X year(s)<br />

% change <strong>in</strong> perceived cost <strong>of</strong><br />

conflict<br />

% change <strong>in</strong> # <strong>of</strong> people who<br />

believe that they will <strong>in</strong>cur<br />

economic losses if violence<br />

breaks out<br />

% change <strong>in</strong> # <strong>of</strong> people who<br />

see violence as a way to earn<br />

money<br />

% change <strong>in</strong> # <strong>of</strong> people who<br />

cite economic reasons for not<br />

support<strong>in</strong>g violence<br />

The top level <strong>in</strong> each <strong>of</strong> the above results cha<strong>in</strong>s is ‘<strong>in</strong>crease <strong>in</strong> peace and stability’. When there are<br />

many factors <strong>in</strong>fluenc<strong>in</strong>g this results, an <strong>in</strong>dex <strong>in</strong>dicator can aggregate changes across multiple drivers <strong>of</strong><br />

conflict. The appropriate <strong>in</strong>dicators to <strong>in</strong>clude will vary based on the context, but could <strong>in</strong>clude:<br />

% reduction <strong>in</strong> perceptions that conflict is likely<br />

% <strong>of</strong> the population who perceive improv<strong>in</strong>g security situation, disaggregated as appropriate<br />

# <strong>of</strong> violent confrontations and extrajudicial kill<strong>in</strong>gs over land, water, or graz<strong>in</strong>g rights (by identity<br />

group)<br />

% change <strong>in</strong> attitudes toward peace and violence<br />

% change <strong>in</strong> perception <strong>of</strong> personal safety<br />

2.2 <strong>Measur<strong>in</strong>g</strong> the universal impact <strong>in</strong>dicators (Control Po<strong>in</strong>t 2.2)<br />

The <strong>in</strong>clusion <strong>of</strong> the three universal impact <strong>in</strong>dicators – scale, <strong>in</strong>come, employment – <strong>in</strong> monitor<strong>in</strong>g<br />

systems supports the aggregation <strong>of</strong> PSD impacts across multiple projects. Their application <strong>in</strong> CAEs is<br />

important, as there is evidence that generat<strong>in</strong>g <strong>in</strong>come and employment can contribute to <strong>in</strong>creased<br />

stability. 40 As with conflict <strong>in</strong>dicators, it is particularly critical to disaggregate employment and <strong>in</strong>come<br />

results to assess the distribution <strong>of</strong> benefits.<br />

There is general consensus the uniqueness <strong>of</strong> each CAE prevents the development <strong>of</strong> universal<br />

<strong>in</strong>dicators for peacebuild<strong>in</strong>g. 41 Rather, other impact-level <strong>in</strong>dicators should be developed as needed<br />

based on the specific nature <strong>of</strong> the conflict. This will be particularly important for PSD projects with a<br />

peacebuild<strong>in</strong>g objective.<br />

40 ECOPA, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict-affected countries, Phase 2: Case studies and<br />

recommendations, August 2011, 24.<br />

41 ECOPA, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict-affected countries, Phase 2: Case studies and<br />

recommendations, August 2011, 106.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

25


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

2.3 Assess<strong>in</strong>g likelihood <strong>of</strong> susta<strong>in</strong>able impact (Control Po<strong>in</strong>t 2.3)<br />

The <strong>in</strong>clusion <strong>of</strong> <strong>in</strong>dicators that assess the susta<strong>in</strong>ability <strong>of</strong> project <strong>in</strong>terventions pushes projects to<br />

assess whether development <strong>in</strong>terventions will lead to last<strong>in</strong>g change. The unpredictability and<br />

fluctuations <strong>in</strong> CAEs makes the use <strong>of</strong> <strong>in</strong>dicators <strong>of</strong> susta<strong>in</strong>ability particularly important. Given the<br />

devastat<strong>in</strong>g impact that conflict can have upon the private sector, monitor<strong>in</strong>g the potential for conflict is<br />

a critical aspect <strong>of</strong> susta<strong>in</strong>ability. The <strong>in</strong>dicators presented above can help <strong>in</strong> accomplish<strong>in</strong>g this.<br />

2.4 Manag<strong>in</strong>g <strong>in</strong>dicators (Control Po<strong>in</strong>t 2.4)<br />

Dur<strong>in</strong>g the course <strong>of</strong> project implementation, changes <strong>in</strong><br />

the CAE context and project learn<strong>in</strong>g based on<br />

implementation results may both require changes to the<br />

results cha<strong>in</strong>s. This may require different or additional<br />

project <strong>in</strong>dicators.<br />

Relevant to<br />

project logic<br />

A common mistake <strong>in</strong> CAEs (and elsewhere) is to<br />

overburden staff with <strong>in</strong>dicators. Ensure <strong>in</strong>dicators are<br />

relevant to the logic <strong>of</strong> your project and that there is<br />

capacity and budget to collect, analyse and use them.<br />

Hard<br />

collect<br />

to<br />

Easy<br />

collect<br />

to<br />

One way to manage this is to plot all current <strong>in</strong>dicators on<br />

a grid like the one to the right, potentially as part <strong>of</strong> a<br />

results measurement workshop. 42 Select <strong>in</strong>dicators <strong>in</strong> the<br />

top right <strong>of</strong> the graph that are relevant to the project<br />

logic and easy to collect while avoid<strong>in</strong>g those <strong>in</strong> the<br />

bottom left.<br />

Irrelevant to<br />

project logic<br />

Preference<br />

Figure 4: Indicator Selection Tool<br />

Tip!<br />

Community members are <strong>of</strong>ten are best placed to understand what would represent an improvement <strong>in</strong><br />

peace <strong>in</strong> their particular context. Runn<strong>in</strong>g workshops with them can be an important source <strong>of</strong><br />

<strong>in</strong>formation <strong>in</strong> the <strong>in</strong>dicator development process. 43 For <strong>in</strong>stance, <strong>in</strong> Nepal a community peacebuild<strong>in</strong>g<br />

project consulted community members to develop <strong>in</strong>dicators for an economic development project.<br />

They learned that be<strong>in</strong>g viewed as credit-worthy by local moneylenders was a critical early <strong>in</strong>dicator that<br />

the ultra poor had made economic ga<strong>in</strong>s. 44<br />

2.5 Projections (Control Po<strong>in</strong>ts 2.5)<br />

Predict<strong>in</strong>g the results <strong>of</strong> programme <strong>in</strong>terventions provides staff with targets and a way to assess<br />

whether <strong>in</strong>terventions are proceed<strong>in</strong>g as expected. They also enable up-front assessments <strong>of</strong> whether<br />

the expected results <strong>of</strong> specific <strong>in</strong>terventions are worth their <strong>in</strong>vestment.<br />

42 RAND Europe, <strong>in</strong>terview September 25, 2012.<br />

43 Sartorius, Rolf and Christopher Carver. Monitor<strong>in</strong>g, Evaluation and Learn<strong>in</strong>g for Fragile States and Peacebuild<strong>in</strong>g<br />

Programs: Practical Tools for Improv<strong>in</strong>g Program Performance and Results, undated, 44.<br />

44 McGrew, Laura, Community Peace build<strong>in</strong>g <strong>in</strong> Nepal: Lessons Learned, 2006, 11.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

26


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Sett<strong>in</strong>g projections <strong>in</strong> a CAE requires mak<strong>in</strong>g adjustments for the difficult operat<strong>in</strong>g environment.<br />

Projects frequently take longer to achieve their goals <strong>in</strong> CAEs. For example, a survey <strong>of</strong> International<br />

F<strong>in</strong>ance Corporation (IFC) projects found that just 17% achieved their timeframe targets <strong>in</strong> environments<br />

where conflict was <strong>in</strong>creas<strong>in</strong>g, compared to approximately 50% <strong>of</strong> projects <strong>in</strong> non-CAEs. 45 To<br />

compensate, projects are advised to set realistic targets. The most common adjustments are to reduce<br />

expected deliverables and/or lengthen the timeframe for achievements. The many delays, disruptions<br />

and setbacks that frequently occur <strong>in</strong> CAEs can be reflected by sett<strong>in</strong>g longer timeframes for<br />

deliverables. Another strategy is to reduce the duration <strong>of</strong> time over which projections are made so as<br />

to account for the fast-chang<strong>in</strong>g environment.<br />

2.6 Key Resources<br />

Agoglia et al. <strong>Measur<strong>in</strong>g</strong> Progress <strong>in</strong> Conflict Environments. Provides a significant volume <strong>of</strong> outcome<br />

<strong>in</strong>dicators around five aspects <strong>of</strong> a conflict, <strong>in</strong>clud<strong>in</strong>g the existence <strong>of</strong> a susta<strong>in</strong>able economy. These are<br />

high-level <strong>in</strong>dicators for gaug<strong>in</strong>g the overall stability <strong>of</strong> a situation, rather than the direct success <strong>of</strong> an<br />

<strong>in</strong>dividual project<br />

CDA. Indications for Assess<strong>in</strong>g Aid’s Impact on Conflict. Offers <strong>in</strong>dicators to measure a project’s<br />

positive and negative effects on conflict.<br />

Church, Cheyanne and Mark M. Roberts. Design<strong>in</strong>g for Results: Integrat<strong>in</strong>g M&E <strong>in</strong> Conflict<br />

Transformation Programmes. Includes general guidel<strong>in</strong>es on monitor<strong>in</strong>g and evaluat<strong>in</strong>g <strong>in</strong> CAEs.<br />

Chapter Four <strong>in</strong>cludes a good discussion on <strong>in</strong>dicators and some helpful examples.<br />

DFID. Interim Guidance Note: <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict-Affected<br />

States and Situations. Lists <strong>in</strong>dicators for high level objectives and possible data sources <strong>in</strong> its Annex A.<br />

More focused on country programmes than projects.<br />

Mercy Corps. Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict & Economics:<br />

Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Provides a menu <strong>of</strong> 29 peace and conflict <strong>in</strong>dicators and 49<br />

economics and conflict <strong>in</strong>dicators. Each <strong>in</strong>dicator is def<strong>in</strong>ed and possible ways to disaggregate the data<br />

are suggested. There is a useful discussion <strong>of</strong> how the <strong>in</strong>dicators were selected and measured.<br />

3 <strong>Measur<strong>in</strong>g</strong> Changes <strong>in</strong> Indicators<br />

It is critical to measure changes <strong>in</strong> <strong>in</strong>dicators to assess project performance and impact. However, there<br />

are many challenges to data collection <strong>in</strong> CAEs. These challenges are frequently present <strong>in</strong> humanitarian<br />

crises and develop<strong>in</strong>g countries more generally, and so may be relevant for practitioners implement<strong>in</strong>g<br />

the <strong>DCED</strong> Standard <strong>in</strong> those contexts. Challenges <strong>in</strong>clude: 46<br />

45 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />

Phase 1: Formative research and key issue diagnosis, undated, 6.<br />

46 For further reference, see DFID, Manag<strong>in</strong>g Results <strong>in</strong> Conflict Affected States, Annex J.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

27


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Limited access. Risks <strong>of</strong> harm to project staff and project clients may limit the ability to access<br />

project sites and meet with project clients. Political <strong>in</strong>stitutions may also not allow access, or<br />

restrict the questions that can be asked.<br />

Weak capacity. It is <strong>of</strong>ten challeng<strong>in</strong>g to f<strong>in</strong>d and reta<strong>in</strong> effective results measurement staff, 47<br />

while low capacity among third parties (universities, survey firms, consultants) may reduce the<br />

ability to outsource the collection <strong>of</strong> data.<br />

Limited secondary data. Government <strong>in</strong>stitutions may not collect or publish accurate data<br />

ow<strong>in</strong>g to limited capacity and resources. In particular, there may be limited data on populations,<br />

which reduces the validity <strong>of</strong> sampl<strong>in</strong>g techniques. This may necessitate an <strong>in</strong>creased amount <strong>of</strong><br />

primary data collection.<br />

Greater monitor<strong>in</strong>g costs. A lack <strong>of</strong> <strong>in</strong>frastructure (e.g. roads, telecommunications networks),<br />

security costs and recruitment challenges will typically require higher spend<strong>in</strong>g on results<br />

measurement. 48<br />

Sensitivity <strong>of</strong> data. Clients and partners may be put at risk if project data is accessed by parties<br />

to the conflict. Obta<strong>in</strong><strong>in</strong>g sensitive <strong>in</strong>formation (e.g. lists <strong>of</strong> beneficiaries, their locations,<br />

household <strong>in</strong>come, assets levels, ethnicity) can expose <strong>in</strong>dividuals to danger or create jealousy<br />

among those who did not benefit. This may also cause unwill<strong>in</strong>gness among beneficiaries and<br />

other stakeholders to share <strong>in</strong>formation. Participants <strong>in</strong> focus group discussions held by Mercy<br />

Corps <strong>in</strong> Indonesia, for <strong>in</strong>stance, requested that it be known that they had not benefited from<br />

their participation <strong>in</strong> the discussion. 49<br />

Data <strong>in</strong>terference. Local staff or partners may be biased <strong>in</strong> their report<strong>in</strong>g. For example, <strong>in</strong> an<br />

area with strong clan loyalties they may feel under pressure to portray their own ethnic group<br />

positively, or to report results that will cause a project to cont<strong>in</strong>ue operat<strong>in</strong>g <strong>in</strong> their home area.<br />

Slower implementation and results. It takes longer to achieve and measure results relative to<br />

other contexts. 50 For example, the Department for International <strong>Development</strong> (DFID) notes that<br />

it can require 15 years to build peace <strong>in</strong> the best <strong>of</strong> situations, which is far longer than the<br />

typical project report<strong>in</strong>g cycle. 51<br />

3.1 Collect<strong>in</strong>g basel<strong>in</strong>e <strong>in</strong>formation (Control Po<strong>in</strong>t 3.1)<br />

Basel<strong>in</strong>es are an essential tool for results measurement. They capture the situation at the beg<strong>in</strong>n<strong>in</strong>g <strong>of</strong><br />

an <strong>in</strong>tervention, enabl<strong>in</strong>g change <strong>in</strong> key <strong>in</strong>dicators to be detected. They can also capture additional<br />

<strong>in</strong>formation about the context, such as further explor<strong>in</strong>g issues that arose dur<strong>in</strong>g conflict analysis.<br />

47 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />

Phase 1: Formative research and key issue diagnosis, undated, 21.<br />

48 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />

Phase 1: Formative research and key issue diagnosis, undated, 20-21.<br />

49 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions: Indonesia Case Study Report,<br />

2011, 9.<br />

50 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />

Phase 1: Formative research and key issue diagnosis, undated, 20.<br />

51 DFID, <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict-Affected States and Situations: How To Note,<br />

2012, 3.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

28


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

However, basel<strong>in</strong>es <strong>in</strong> CAEs can be particularly complex due to the challenges listed above. Moreover,<br />

projects <strong>in</strong> CAEs are more likely to require a flexible implementation strategy that <strong>in</strong>volves chang<strong>in</strong>g<br />

their activities, outputs and outcomes as the project goes on. This could conceivably render orig<strong>in</strong>al<br />

basel<strong>in</strong>e <strong>in</strong>formation irrelevant. The <strong>DCED</strong> Standard emphasizes the importance <strong>of</strong> articulat<strong>in</strong>g specific<br />

results cha<strong>in</strong>s and <strong>in</strong>dicators for each project <strong>in</strong>tervention. Consequently, <strong>in</strong>stead <strong>of</strong> conduct<strong>in</strong>g a s<strong>in</strong>gle,<br />

large project-wide basel<strong>in</strong>e, each <strong>in</strong>tervention may have its own smaller basel<strong>in</strong>e. This enables projects<br />

to avoid discard<strong>in</strong>g all basel<strong>in</strong>e data if, for <strong>in</strong>stance, violent upheaval renders some <strong>of</strong> a project’s<br />

<strong>in</strong>terventions unfeasible.<br />

Retrospective basel<strong>in</strong>es are basel<strong>in</strong>es conducted after the <strong>in</strong>tervention has already started, by ask<strong>in</strong>g<br />

participants to recall the situation before the start <strong>of</strong> the project. They may be the only option to collect<br />

<strong>in</strong>formation about the past, although their use is not ideal and can produce biased and erroneous<br />

results. 52 If us<strong>in</strong>g retrospective basel<strong>in</strong>es, it is important to carefully consider both the time lag s<strong>in</strong>ce<br />

the event <strong>in</strong> question has occurred and the type <strong>of</strong> event that you are ask<strong>in</strong>g about. The type <strong>of</strong><br />

experience determ<strong>in</strong>es how well it can be recalled. For example, questions about actions performed<br />

directly by the respondent are more likely to be recalled. Information that was heard or read – for<br />

example, market access <strong>in</strong>formation provided by a service provider <strong>in</strong> a phone call – will be forgotten<br />

more quickly. A rough guide is to not ask recall questions about events that occurred over a year ago, as<br />

memory becomes <strong>in</strong>creas<strong>in</strong>gly unreliable.<br />

3.2 Good research practices (Control Po<strong>in</strong>t 3.2)<br />

While the standard good research practices that are listed <strong>in</strong> the <strong>DCED</strong>’s Guide to Conduct<strong>in</strong>g Research 53<br />

apply to CAEs, there are other important considerations. The challenges <strong>of</strong> work<strong>in</strong>g <strong>in</strong> CAEs mean that<br />

the acceptable standard for research practices, <strong>in</strong>clud<strong>in</strong>g sample sizes and attribution measurement,<br />

(see the section on attribution, below) may be less rigorous. Less exact<strong>in</strong>g standards <strong>of</strong> practice are<br />

sometimes all that is possible.<br />

This section presents good practices for design<strong>in</strong>g research tools and collection <strong>in</strong>formation <strong>in</strong> CAEs.<br />

Ultimately, ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>g a focus on the three pr<strong>in</strong>ciples for success (simplicity, flexibility, sensitivity) is<br />

key.<br />

Some tips for select<strong>in</strong>g questions and <strong>in</strong>formation tools are:<br />

Use short, simple questions. Simple questions are more easily understood by staff and users<br />

and therefore provide better results. Overly complicated questions can greatly <strong>in</strong>crease the<br />

workload both <strong>in</strong> data collection and analysis. Similarly, close-ended survey questions are <strong>of</strong>ten<br />

preferable for their greater ease <strong>of</strong> cod<strong>in</strong>g and analys<strong>in</strong>g. Focus group discussions and surveys<br />

also should be <strong>of</strong> a manageable size. One practitioner suggests that 40 survey questions and 10<br />

focus group discussion questions should be the maximum <strong>in</strong> order to not overwhelm data<br />

52 Ravaillon, Mart<strong>in</strong>, Can We Trust Shoestr<strong>in</strong>g Evaluations Policy Research Work<strong>in</strong>g Paper 5983, 2012.<br />

53 Donor Committee for Enterprise <strong>Development</strong>, <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong><br />

Implementation Guidel<strong>in</strong>es, Version 1g, 5 th March 2010, 26-27.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

29


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

collection teams and respondents. 54 Shorter surveys also typically allow a greater number <strong>of</strong><br />

<strong>in</strong>terviews to be conducted and thus <strong>in</strong>crease the total potential sample size.<br />

Complement the measurement <strong>of</strong> beliefs and attitudes when possible. Ask<strong>in</strong>g about beliefs<br />

and attitudes can provide helpful <strong>in</strong>formation about changes that are otherwise difficult to<br />

measure. However beliefs and attitudes can be <strong>in</strong>fluenced by many factors, can change based<br />

on events occurr<strong>in</strong>g around the time <strong>of</strong> the survey and may be tailored to fit the expected biases<br />

<strong>of</strong> the <strong>in</strong>terviewers. Ask<strong>in</strong>g about behaviour can help to complement this <strong>in</strong>formation. For<br />

example, ask<strong>in</strong>g about levels <strong>of</strong> trust between groups directly may not elicit useful responses.<br />

Behavioural questions such as “<strong>in</strong> the last year, have you bought anyth<strong>in</strong>g from a member <strong>of</strong><br />

group X” can be more reliable.<br />

Use proxy <strong>in</strong>dicators as necessary to be conflict sensitive. In some cases, it might be dangerous<br />

or <strong>in</strong>appropriate to directly ask about conflict related factors. In this case, you can use proxy<br />

<strong>in</strong>dicators. For <strong>in</strong>stance, the breadth <strong>of</strong> <strong>in</strong>ter-group economic transactions, rate <strong>of</strong><br />

<strong>in</strong>termarriage or collaboration on decision-mak<strong>in</strong>g bodies are all potential proxy <strong>in</strong>dicators to<br />

measure the state <strong>of</strong> <strong>in</strong>ter-group acceptance. 55<br />

Field test tools. It is important to test tools with staff, partners and community members before<br />

widespread roll out. Many methods look easy on paper but require f<strong>in</strong>e tun<strong>in</strong>g once you start to<br />

use them. Field-test<strong>in</strong>g tools will help ensure that questions are appropriate, conflict-sensitive,<br />

and simple to adm<strong>in</strong>ister.<br />

Some tips for data-collection are:<br />

Build trust. Results measurement frequently relies on discussions or surveys with those affected<br />

by the <strong>in</strong>tervention, such as partners and the communities who utilise products or services. If<br />

these groups do not trust the project staff, they may refuse to answer questions or choose not<br />

to answer honestly. This is especially relevant when <strong>in</strong>vestigat<strong>in</strong>g sensitive topics related to<br />

conflict. Ask<strong>in</strong>g for direct feedback on project impact may also be challeng<strong>in</strong>g, as this will be<br />

perceived to affect future fund<strong>in</strong>g <strong>in</strong> the area. Consequently, programmes should build strong<br />

relationships with partners and the community.<br />

Provide tra<strong>in</strong><strong>in</strong>g <strong>in</strong> conflict-sensitive monitor<strong>in</strong>g approaches. To mitigate the risk <strong>of</strong> <strong>in</strong>flam<strong>in</strong>g<br />

sensitivities through data collection, adequately tra<strong>in</strong> the <strong>in</strong>dividuals collect<strong>in</strong>g monitor<strong>in</strong>g<br />

<strong>in</strong>formation. Tra<strong>in</strong><strong>in</strong>g should emphasize provid<strong>in</strong>g the option for participants to not answer<br />

questions if they wish, keep<strong>in</strong>g answers confidential, and tak<strong>in</strong>g care when ask<strong>in</strong>g questions<br />

which may be sensitive. In certa<strong>in</strong> contexts the use <strong>of</strong> terms like conflict or clan are highly<br />

<strong>in</strong>flammatory, while <strong>in</strong> others – such as was the experience <strong>of</strong> Mercy Corps <strong>in</strong> Uganda 56 – they<br />

are not.<br />

54 Interview with Jenny Vaughan (Mercy Corps), October 1, 2012.<br />

55 DFID, Work<strong>in</strong>g Effectively <strong>in</strong> Conflict-affected and Fragile Situations: Brief<strong>in</strong>g Paper I: Monitor<strong>in</strong>g and Evaluation,<br />

9.<br />

56 Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions: Uganda Case Study Report,<br />

2011, 9.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

30


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Use mobile technologies. Technology can reduce or elim<strong>in</strong>ate the need for physical visits to<br />

collect <strong>in</strong>formation, thereby reduc<strong>in</strong>g both cost and risk. Mobile phones are common and cheap<br />

<strong>in</strong> most CAEs, and <strong>of</strong>ten provide a good technological solution to <strong>in</strong>formation gather<strong>in</strong>g. 57<br />

Case Study: Collect<strong>in</strong>g Conflict-Related Data through SMS Report<strong>in</strong>g 58<br />

Columbia University’s Voix-des-Kivus project piloted a data-gather<strong>in</strong>g system <strong>in</strong> Eastern Congo <strong>in</strong> which<br />

reporters <strong>in</strong> randomly selected villages reported on events <strong>in</strong> real time. They selected three reporters <strong>in</strong><br />

each <strong>of</strong> 18 villages; one traditional leader, one women’s representative, and one elected by the<br />

community. Each received $1.5 a week <strong>in</strong> return for send<strong>in</strong>g at least one text report<strong>in</strong>g on current<br />

events. A pre-agreed cod<strong>in</strong>g system allowed for easy and systemised data collection. For example, a text<br />

beg<strong>in</strong>n<strong>in</strong>g ‘46’ <strong>in</strong>dicated ethnic violence.<br />

In total, 4,623 unique events were reported – many were reported by more than one person. 30% <strong>of</strong><br />

these were conflict-related. The researchers found that this method was much cheaper and more<br />

reliable than traditional questionnaire data. It works <strong>in</strong> places where survey teams cannot reach, gathers<br />

<strong>in</strong>formation <strong>in</strong> real time, and builds a relationship <strong>of</strong> trust between the project and the reporters,<br />

enabl<strong>in</strong>g higher-quality <strong>in</strong>formation to be collected. By comb<strong>in</strong><strong>in</strong>g this rich dataset with a randomised<br />

<strong>in</strong>tervention run by the IRC, Columbia University found aid projects reduced conflict. There is the<br />

potential for a PSD project to imitate aspects <strong>of</strong> this pilot. It improves accountability to the local<br />

community, allow<strong>in</strong>g them to text suggestions and compla<strong>in</strong>ts. Moreover, it can be <strong>in</strong>tegrated <strong>in</strong>to the<br />

results measurement system to monitor the effects <strong>of</strong> the project on conflict <strong>in</strong> real time.<br />

Build flexibility <strong>in</strong>to the monitor<strong>in</strong>g schedule. Schedule additional time for data collection to<br />

allow for potential disruptions. Develop an alternate survey plan with a second sample already<br />

selected <strong>in</strong> case security issues prevent the <strong>in</strong>itial plan from proceed<strong>in</strong>g.<br />

Select a conflict-appropriate sample. Depend<strong>in</strong>g upon the type <strong>of</strong> <strong>in</strong>formation you wish to<br />

collect and conflict-related sensitivities, a purposive sample may be preferable to a random<br />

sample. A purposive sample can allow you to capture data from perspectives that are important<br />

to the conflict but that may be missed by us<strong>in</strong>g a randomized approach.<br />

Build a balanced monitor<strong>in</strong>g team. Us<strong>in</strong>g a mix <strong>of</strong> local and non-local staff br<strong>in</strong>gs both an<br />

<strong>in</strong>dependent perspective and local knowledge to monitor<strong>in</strong>g. This can br<strong>in</strong>g <strong>in</strong> external<br />

expertise and reduce bias <strong>in</strong> data collection while benefit<strong>in</strong>g from local knowledge and<br />

experience. It can also build the capacity <strong>of</strong> the local staff if well managed.<br />

Protect aga<strong>in</strong>st loss <strong>of</strong> your sample. In contexts where migration is high and where there is<br />

<strong>in</strong>security, projects may be unable to re-survey some members <strong>of</strong> their basel<strong>in</strong>e. Ways to<br />

protect aga<strong>in</strong>st the loss <strong>of</strong> one’s sample <strong>in</strong>clud<strong>in</strong>g oversampl<strong>in</strong>g (i.e. survey<strong>in</strong>g a larger<br />

population than is statistically required) to ga<strong>in</strong> a larger <strong>in</strong>itial sample and request<strong>in</strong>g a contact<br />

57 One popular free s<strong>of</strong>tware for conduct<strong>in</strong>g polls and surveys through mobile phone is Frontl<strong>in</strong>e SMS.<br />

58 W<strong>in</strong>dt and Humphreys, Crowdseed<strong>in</strong>g Conflict Data: An Application <strong>of</strong> an SMS-based Data System to Estimate<br />

the Conflict Effects <strong>of</strong> <strong>Development</strong> Aid<br />

W<strong>in</strong>dt and Humphreys, Voix des Kivus: Reflections on a Crowdseed<strong>in</strong>g Approach to Conflict Event Data Gather<strong>in</strong>g,<br />

2012.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

31


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

person who would know where participants have gone if they relocate. The IRC’s large sample<br />

size across multiple regions <strong>in</strong> the Democratic Republic <strong>of</strong> the Congo allowed a randomized<br />

control trial to cont<strong>in</strong>ue despite political issues <strong>in</strong> one region that ended its data collection<br />

there. 59 The IRC has also found that the basel<strong>in</strong>e data collected on <strong>in</strong>dividuals who leave can<br />

still serve the purpose <strong>of</strong> compar<strong>in</strong>g the characteristics <strong>of</strong> more mobile populations versus those<br />

who stayed <strong>in</strong> the program.<br />

Triangulate <strong>in</strong>formation. Given the frequent lack <strong>of</strong> high quality data <strong>in</strong> a CAE, all <strong>in</strong>formation<br />

should be triangulated where feasible. This means that multiple methods should be used to<br />

improve the reliability <strong>of</strong> the results. Often qualitative, quantitative and participatory methods<br />

can be used together to triangulate <strong>in</strong>formation, as can <strong>in</strong>terview<strong>in</strong>g multiple sources.<br />

Triangulat<strong>in</strong>g <strong>in</strong>formation <strong>in</strong> the SEED programme<br />

The SEED programme <strong>in</strong> Somalia, implemented by FAO, uses a variety <strong>of</strong> ways to triangulate<br />

<strong>in</strong>formation, <strong>in</strong>clud<strong>in</strong>g:<br />

Partner reports. Implement<strong>in</strong>g NGOs are required to submit regular reports on their progress.<br />

Random calls. A dedicated unit with<strong>in</strong> FAO calls a random sample <strong>of</strong> between 2-3% <strong>of</strong> all<br />

beneficiaries, to verify that reported results have been achieved. They also call key <strong>in</strong>formants,<br />

such as elders or market venders.<br />

Field monitors. The monitor<strong>in</strong>g unit has 25 local field monitors, who can access <strong>in</strong>secure areas<br />

and verify that the activities are tak<strong>in</strong>g place<br />

Satellite imagery. FAO have a large programme to rehabilitate irrigation canals. They require<br />

partners to map out the planned canal route with a GPS device, and FAO use satellite imagery, to<br />

assess the progress <strong>of</strong> the project.<br />

Biometrics. FAO are trial<strong>in</strong>g biometric identification track<strong>in</strong>g via f<strong>in</strong>gerpr<strong>in</strong>t scanners for cash-forwork<br />

programme recipients.<br />

More <strong>in</strong>formation is available <strong>in</strong> the SEED case study, available on the <strong>DCED</strong> website.<br />

Outsource data collection. Outsourc<strong>in</strong>g data collection to local entities can allow implement<strong>in</strong>g<br />

agencies to collect data <strong>in</strong> areas where implement<strong>in</strong>g agency staff lack access. It can also reduce<br />

the political pressure on project implementers to report positive results by creat<strong>in</strong>g a formal<br />

barrier <strong>in</strong> data collection. Effective outsourc<strong>in</strong>g does however require effective oversight by the<br />

implement<strong>in</strong>g agency.<br />

3.3 Qualitative <strong>in</strong>formation (Control Po<strong>in</strong>t 3.3)<br />

Qualitative <strong>in</strong>formation is necessary <strong>in</strong> any context, but especially <strong>in</strong> CAEs, where perceptions, beliefs<br />

and op<strong>in</strong>ions are essential to the success <strong>of</strong> a project. Qualitative <strong>in</strong>formation typically requires more<br />

time to collect and analyse than qualitative <strong>in</strong>formation. It is therefore important to carefully consider<br />

when qualitative question<strong>in</strong>g is most important and use it judiciously. Too many qualitative questions<br />

on surveys, for <strong>in</strong>stance, <strong>in</strong>creases the time required for <strong>in</strong>terviews and subsequent analysis. This may<br />

59 Interview with IRC, Jeannie Annan, September 26, 2012.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

32


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

limit your sample size and therefore your ability to extrapolate from your f<strong>in</strong>d<strong>in</strong>gs to the broader<br />

population.<br />

One common approach to collect<strong>in</strong>g qualitative <strong>in</strong>formation is to use participatory techniques that take<br />

<strong>in</strong>to account the perspectives and <strong>in</strong>sights <strong>of</strong> all stakeholders, beneficiaries and partners as well as<br />

project implementers. They typically emphasise build<strong>in</strong>g the capacity <strong>of</strong> all these groups to reflect on<br />

<strong>in</strong>terventions to analyse what worked and did not. This can be used to strengthen relationships among<br />

parties to a conflict and agreement on future collaboration.<br />

3.4 Key resources<br />

Mercy Corps, Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions: Conflict & Economics:<br />

Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Provides several conflict monitor<strong>in</strong>g tools, <strong>in</strong>clud<strong>in</strong>g a violent<br />

<strong>in</strong>cident report<strong>in</strong>g form, a disputes and dispute resolution assessment, a scored community relationship<br />

mapp<strong>in</strong>g tool, and a conflict and resource mapp<strong>in</strong>g tool.<br />

Catley, Burns, Abebe and Suji, Participatory Impact Assessment: A Guide for Practitioners, 2008,<br />

Fe<strong>in</strong>ste<strong>in</strong> Centre. Outl<strong>in</strong>es an eight-step approach to conduct<strong>in</strong>g a participatory impact assessment.<br />

Frontl<strong>in</strong>e SMS. A free s<strong>of</strong>tware for creat<strong>in</strong>g and manag<strong>in</strong>g SMS activities such as mak<strong>in</strong>g<br />

announcements, conduct<strong>in</strong>g polls, and automatically reply<strong>in</strong>g to <strong>in</strong>com<strong>in</strong>g SMS. In particular, it is<br />

excellent for manag<strong>in</strong>g large mobile phone surveys and was used <strong>in</strong> the above example by Colombia<br />

University to gather data <strong>in</strong> Eastern DRC.<br />

Muaz Jalil. Practical Guidel<strong>in</strong>es for Conduct<strong>in</strong>g Research. Provides a review <strong>of</strong> good practices <strong>in</strong><br />

design<strong>in</strong>g and conduct<strong>in</strong>g research <strong>in</strong> l<strong>in</strong>e with the <strong>DCED</strong> Standard.<br />

Sartorius, Rolf and Christopher Carver. Monitor<strong>in</strong>g, Evaluation and Learn<strong>in</strong>g for Fragile States and<br />

Peacebuild<strong>in</strong>g Programs: Practical Tools for Improv<strong>in</strong>g Program Performance and Results. Offers a<br />

large number <strong>of</strong> monitor<strong>in</strong>g tools for peacebuild<strong>in</strong>g programmes, tips on measurement and sample<br />

<strong>in</strong>dicators.<br />

4 Estimat<strong>in</strong>g Attributable Changes<br />

Estimat<strong>in</strong>g attribution requires establish<strong>in</strong>g a causal l<strong>in</strong>k between an <strong>in</strong>tervention and observed changes<br />

<strong>in</strong> <strong>in</strong>dicators and account<strong>in</strong>g for what would have happened even if the program had not been operat<strong>in</strong>g<br />

(i.e. the counterfactual). This is difficult because many other factors may contribute to the change. For<br />

<strong>in</strong>stance, a 50% <strong>in</strong>crease <strong>in</strong> employment among project sites <strong>in</strong> Somalia follow<strong>in</strong>g an <strong>in</strong>tervention may<br />

be due to the project’s activities but could also be ow<strong>in</strong>g to favourable macroeconomic conditions, an<br />

<strong>in</strong>crease <strong>in</strong> illicit activities (lead<strong>in</strong>g to <strong>in</strong>creased wealth and employment), or good weather. To estimate<br />

attributable change, it is necessary to assess how much <strong>of</strong> the observed results are caused by the<br />

project and how much would have happened without it.<br />

Design<strong>in</strong>g an effective approach to attribution is essential to properly assess<strong>in</strong>g the impact <strong>of</strong> your<br />

project, as it greatly <strong>in</strong>creases the credibility <strong>of</strong> reported results. While always a challenge, <strong>in</strong> CAEs<br />

design<strong>in</strong>g an effective approach to attribution is particularly difficult because <strong>of</strong>:<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

33


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Difficulty <strong>of</strong> <strong>in</strong>formation gather<strong>in</strong>g. Prov<strong>in</strong>g attribution <strong>of</strong>ten requires gather<strong>in</strong>g additional<br />

<strong>in</strong>formation, which is particularly challeng<strong>in</strong>g <strong>in</strong> a CAE. This is particularly problematic for<br />

experimental or quasi-experimental designs that require large amounts <strong>of</strong> high quality data.<br />

Rapidly chang<strong>in</strong>g context. CAEs typically change rapidly. If conflict is ongo<strong>in</strong>g, then conditions<br />

may be rapidly deteriorat<strong>in</strong>g. If it is post-conflict, the environment may be rapidly improv<strong>in</strong>g as<br />

the country moves forward from the war. In both cases, relationships between different groups,<br />

the bus<strong>in</strong>ess environment, and external <strong>in</strong>vestment will likely be fluctuat<strong>in</strong>g greatly. This makes<br />

it harder to attribute change to your programme.<br />

More complex theories <strong>of</strong> change. The more external factors affect the outcomes you are<br />

measur<strong>in</strong>g, the harder it will be to attribute change. This is because there will be more<br />

alternative explanations for the change that you observe, and so it will be more challeng<strong>in</strong>g to<br />

show the role played by your programme. A sub-national conflict, for <strong>in</strong>stance, may be<br />

<strong>in</strong>fluenced by <strong>in</strong>ternational policies (e.g. sanctions), national policies, socio-cultural norms and<br />

many other factors. There are usually multiple causes <strong>of</strong> any change and disentangl<strong>in</strong>g the l<strong>in</strong>ks<br />

between them is extremely difficult. Whereas many PSD projects attempt simply to attribute<br />

positive economic outcomes (e.g. employment, <strong>in</strong>come) for project clients to their<br />

<strong>in</strong>terventions, those with peacebuild<strong>in</strong>g objectives <strong>of</strong>ten try to establish that those positive<br />

economic outcomes reduced or prevented conflict. The greater number <strong>of</strong> l<strong>in</strong>ks to be proven<br />

<strong>in</strong>creases the challenge <strong>of</strong> demonstrat<strong>in</strong>g attribution.<br />

The <strong>DCED</strong> Standard recognises the complexity <strong>of</strong> demonstrat<strong>in</strong>g attribution, and advocates for an<br />

approach that would conv<strong>in</strong>ce a “reasonable but sceptical observer”. 60 It recognises that there are<br />

multiple contributors to any positive (or negative) outcome, and the programme <strong>in</strong> some cases may<br />

have a relatively m<strong>in</strong>or part to play. In particular, the <strong>DCED</strong> Standard does not require ‘scientific’ pro<strong>of</strong><br />

<strong>of</strong> causality, which <strong>of</strong>ten requires randomised control trials and is generally not practical with<strong>in</strong> the<br />

constra<strong>in</strong>ts <strong>of</strong> development programmes. The <strong>DCED</strong> has published separate guidel<strong>in</strong>es on measur<strong>in</strong>g<br />

attribution that can be found here.<br />

4.1 A clear and appropriate system for attribution is <strong>in</strong> place (Control Po<strong>in</strong>t<br />

4.1)<br />

There are three key complementary approaches to estimat<strong>in</strong>g attribution <strong>in</strong> CAEs.<br />

First, use the results cha<strong>in</strong>s as the basis for attribution by validat<strong>in</strong>g each key change. This is a ‘theorybased’<br />

approach to assess<strong>in</strong>g attribution, which argues that there are several necessary conditions for<br />

claim<strong>in</strong>g attribution: 61<br />

The results cha<strong>in</strong> must be well designed, logical and evidence-based.<br />

The expected changes at every level <strong>of</strong> the results cha<strong>in</strong> must have occurred.<br />

Alternative explanations for these changes have been considered and rejected.<br />

60 Sen, Nabanita, Attribution: <strong>Measur<strong>in</strong>g</strong> attributable change caused by a programme, 2012.<br />

61 White and Phillips, Address<strong>in</strong>g attribution <strong>of</strong> cause and effect <strong>in</strong> small n impact evaluations: towards an<br />

<strong>in</strong>tegrated framework, 2012, 3ie, 14<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

34


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

If these are all met, it is reasonable to conclude that the results can be attributed to the project. The<br />

results cha<strong>in</strong> tells a ‘contribution story’ which shows how you believe you <strong>in</strong>fluenced the outcomes.<br />

Second, utilise qualitative tools for assess<strong>in</strong>g attribution. These support the use <strong>of</strong> the results cha<strong>in</strong> to<br />

demonstrate attribution, by assess<strong>in</strong>g the l<strong>in</strong>ks between the results cha<strong>in</strong>. In particular, qualitative tools<br />

are typically well suited for CAE environments <strong>in</strong> which large-scale data collection is challeng<strong>in</strong>g.<br />

Participative methods are one way to assess the difference made by the project. For example, it would<br />

be possible to ask beneficiaries to rank the different factors that contribute towards an observed<br />

outcome. The follow<strong>in</strong>g figure shows one example <strong>in</strong> which participants were asked to rank the factors<br />

contribut<strong>in</strong>g to an improvement <strong>in</strong> food security. Factors coloured <strong>in</strong> blue were <strong>in</strong>fluenced by the<br />

project’s <strong>in</strong>terventions, while those <strong>in</strong> red were not: 62<br />

Figure 5: Factors Reportedly Contribut<strong>in</strong>g to Improved Food Security<br />

In this case, the programmes contributed to the use <strong>of</strong> improved seeds and fertilizer. However, they<br />

cannot claim responsibility for government extension services, and certa<strong>in</strong>ly not for improved ra<strong>in</strong>fall.<br />

Be aware <strong>of</strong> the biases that qualitative and quantitative research is susceptible to; for example,<br />

beneficiaries may wish to give a positive impression <strong>of</strong> the programme <strong>in</strong> order to please the<br />

<strong>in</strong>terviewer, or <strong>in</strong> order to ga<strong>in</strong> more fund<strong>in</strong>g. Likewise, the <strong>in</strong>terviewer may be biased towards their<br />

own organisation.<br />

F<strong>in</strong>ally, quantitative attribution analysis can be applied, <strong>in</strong>clud<strong>in</strong>g techniques like randomised control<br />

trials (RCTs), statistical analysis, or quasi-experimental techniques. They generally rely on collect<strong>in</strong>g<br />

quantitative data from large numbers <strong>of</strong> people, and us<strong>in</strong>g statistical techniques to create a<br />

‘counterfactual’, to understand what the situation would have been like without the <strong>in</strong>tervention. This is<br />

compared to the measured results <strong>of</strong> the <strong>in</strong>tervention to determ<strong>in</strong>e the improvement that can be<br />

attributed to the project. This is <strong>of</strong>ten seen as impractical <strong>in</strong> conflict affected environments, but it can be<br />

achieved.<br />

For example, the International Rescue Committee (IRC) is implement<strong>in</strong>g RCTs <strong>in</strong> Afghanistan, Burundi,<br />

DRC, and Liberia, <strong>in</strong>clud<strong>in</strong>g one <strong>of</strong> a sav<strong>in</strong>gs group programme. IRC’s experience suggests that RCTs are<br />

possible to implement <strong>in</strong> CAEs but face additional challenges. For <strong>in</strong>stance, the lack <strong>of</strong> population data<br />

62 Taken from Catley, Andrew et al, Participatory Impact Assessment - A Guide for Practitioners, Fe<strong>in</strong>ste<strong>in</strong><br />

International Centre, 2008, 50.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

35


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

means that select<strong>in</strong>g an appropriate sample can difficult. Census data must <strong>in</strong> some cases be collected<br />

directly. Further, randomization can <strong>in</strong>crease costs and complexity. In Liberia, for <strong>in</strong>stance, random<br />

selection <strong>of</strong> project sites requires the project to work across areas us<strong>in</strong>g two different languages and<br />

with significant distances between sites. This is particularly an issue when seek<strong>in</strong>g to measure changes <strong>in</strong><br />

conflict at a community level rather than an <strong>in</strong>dividual level: hav<strong>in</strong>g a sufficiently large sample for<br />

randomization would require work<strong>in</strong>g <strong>in</strong> several hundred villages. Another consideration <strong>in</strong> us<strong>in</strong>g a<br />

randomization approach <strong>in</strong> CAEs is that mak<strong>in</strong>g changes to project strategy – a more frequent necessity<br />

<strong>in</strong> such environments – can compromise the f<strong>in</strong>d<strong>in</strong>gs <strong>of</strong> the RCT. There is also a risk that randomization<br />

can exacerbate the conflict by creat<strong>in</strong>g resentment among control groups members <strong>of</strong> the treatment<br />

group. IRC’s experience suggests that this latter challenge can be overcome by be<strong>in</strong>g open and<br />

transparent about the process <strong>of</strong> select<strong>in</strong>g project clients. 63 Still, randomization approaches require<br />

highly tra<strong>in</strong>ed staff and high costs, and will not be realistic for most projects.<br />

In summary, there are several approaches to assess<strong>in</strong>g attribution. The most appropriate comb<strong>in</strong>ation <strong>of</strong><br />

approaches will vary based on the nature <strong>of</strong> the <strong>in</strong>tervention and capacity <strong>of</strong> the implement<strong>in</strong>g team.<br />

4.2 Key resources<br />

Mayne, John. Contribution Analysis: An approach to explor<strong>in</strong>g cause and effect, 2008.<br />

<strong>in</strong>troduction to Contribution Analysis.<br />

A short<br />

Catley, Andrew et al, Participatory Impact Assessment - A Guide for Practitioners, Fe<strong>in</strong>ste<strong>in</strong><br />

International Centre, 2008<br />

OECD. Evaluat<strong>in</strong>g Peacebuild<strong>in</strong>g Activities <strong>in</strong> Sett<strong>in</strong>gs <strong>of</strong> Conflict and Fragility: Improv<strong>in</strong>g Learn<strong>in</strong>g for<br />

Results. Provides guidance on assess<strong>in</strong>g attribution <strong>in</strong> the evaluation <strong>of</strong> peacebuild<strong>in</strong>g activities.<br />

Sen, Nabanita. Attribution: <strong>Measur<strong>in</strong>g</strong> Attributable Change Caused by a Programme. A brief,<br />

accessible outl<strong>in</strong>e <strong>of</strong> how to estimate attribution when apply<strong>in</strong>g the <strong>DCED</strong> Standard.<br />

Stern, Elliot et al. Broaden<strong>in</strong>g the Range <strong>of</strong> Designs and Methods for Impact Evaluations. Addresses<br />

the issue <strong>of</strong> attribution <strong>in</strong> depth, look<strong>in</strong>g at the role <strong>of</strong> mixed methods and theory-based approaches to<br />

assess<strong>in</strong>g attribution.<br />

5 Captur<strong>in</strong>g Wider Changes <strong>in</strong> the System or Market (Control Po<strong>in</strong>ts<br />

5.1)<br />

Systemic change refers to “[c]hanges <strong>in</strong> market systems and the structures, such as government and civil<br />

society, that support markets that cause susta<strong>in</strong>able shifts <strong>in</strong> the way those market systems and<br />

structures operate, for example, changes <strong>in</strong> relationships with<strong>in</strong> and among both private enterprises and<br />

public agencies, <strong>in</strong> <strong>in</strong>centives and <strong>in</strong> market support structures. Systemic change causes widespread<br />

<strong>in</strong>direct results such as crowd<strong>in</strong>g <strong>in</strong>, copy<strong>in</strong>g, enterprises shift<strong>in</strong>g sectors and changes <strong>in</strong> enterprise start-<br />

63 DFID, Interim Guidance Note: <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict-Affected States and<br />

Situations, Undated, 38.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

36


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

up and exit rates.” 64 <strong>Measur<strong>in</strong>g</strong> systemic change is extremely important to understand<strong>in</strong>g the full<br />

impacts <strong>of</strong> PSD programm<strong>in</strong>g. Its omission from results measurement frameworks means that projects<br />

will underreport their impact and miss many <strong>of</strong> the most important types <strong>of</strong> change.<br />

Ow<strong>in</strong>g to the limited experience <strong>in</strong> explicitly apply<strong>in</strong>g the entire <strong>DCED</strong> Standard <strong>in</strong> CAEs, there are still<br />

very few experiences <strong>in</strong> attempt<strong>in</strong>g to measure systemic change <strong>in</strong> a CAE context. Systemic change is<br />

less likely to occur where trust among neighbours and bus<strong>in</strong>esses has been weakened and economic<br />

activity is more limited. Estimations <strong>of</strong> the amount <strong>of</strong> copy<strong>in</strong>g and crowd<strong>in</strong>g-<strong>in</strong> sparked by project<br />

<strong>in</strong>terventions, particularly if derived from benchmarks <strong>in</strong> non-CAEs (e.g. economic multipliers) thus may<br />

need to be more conservative.<br />

6 Track<strong>in</strong>g Programme Costs (Control Po<strong>in</strong>ts 6.1 – 6.2)<br />

Track<strong>in</strong>g programm<strong>in</strong>g costs is important to understand and improve programm<strong>in</strong>g efficiency. In CAEs<br />

characterized by significant pressure to spend quickly, track<strong>in</strong>g costs is particularly important. However,<br />

projects <strong>in</strong> these contexts face a number <strong>of</strong> challenges. The costs <strong>of</strong> implementation are frequently<br />

higher yet achievements (e.g. employment generation, <strong>in</strong>creases <strong>in</strong> <strong>in</strong>come) fewer. 65 As such, be careful<br />

<strong>in</strong> benchmark<strong>in</strong>g the benefit-cost ratios <strong>of</strong> projects implemented <strong>in</strong> CAEs with those implemented<br />

elsewhere. Projects also need to be careful if publish<strong>in</strong>g f<strong>in</strong>ancial reports disaggregated by region or<br />

ethnic group. If spend<strong>in</strong>g is unequally distributed among conflict<strong>in</strong>g areas or groups, this may fuel<br />

tension and put project staff or partners at an <strong>in</strong>creased risk.<br />

7 Report<strong>in</strong>g Results (Control Po<strong>in</strong>ts 7.1 – 7.5)<br />

Report<strong>in</strong>g results creates transparency about development programm<strong>in</strong>g. Given the high visibility <strong>of</strong><br />

many CAEs and pressure from donors to show results, there is <strong>of</strong>ten great pressure to report results<br />

quickly. However, report<strong>in</strong>g project results can potentially create harm <strong>in</strong> CAEs. Identify<strong>in</strong>g specific<br />

clients creates the risk that they can be s<strong>in</strong>gled out for harm, even <strong>in</strong> seem<strong>in</strong>gly safe places like project<br />

reports.<br />

The pressure to report results can also have negative effects on the programme, encourag<strong>in</strong>g short term<br />

<strong>in</strong>terventions and ‘quick w<strong>in</strong>s’ at the expense <strong>of</strong> susta<strong>in</strong>able, longer-term change. Although this is<br />

unavoidable to some extent, programme staff should be aware <strong>of</strong> this pressure and manage<br />

relationships with donors to give themselves adequate time to achieve change <strong>in</strong> a CAE.<br />

Caution!<br />

Carefully consider how to safeguard monitor<strong>in</strong>g <strong>in</strong>formation and publish it <strong>in</strong> responsible ways that do<br />

not put others at risk.<br />

64 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />

<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 18.<br />

65 Dalberg Global <strong>Development</strong> Advisors, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries,<br />

Phase 1: Formative research and key issue diagnosis, undated, 5.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

37


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Consider the follow<strong>in</strong>g when safely report<strong>in</strong>g results:<br />

Data protection: Computers with sensitive data should be password protected, data encrypted<br />

and guarded. 66<br />

Pre-publish<strong>in</strong>g review: All monitor<strong>in</strong>g documentation should be reviewed with a conflictsensitive<br />

perspective prior to publication to ensure that it does not create the risk <strong>of</strong> harm.<br />

Strategies to achieve this can <strong>in</strong>clude seek<strong>in</strong>g permission from any person who is named <strong>in</strong> a<br />

report (<strong>in</strong>clud<strong>in</strong>g staff), develop<strong>in</strong>g guidel<strong>in</strong>es with key stakeholders (e.g. government if<br />

appropriate, donors, partners, local community), or ask<strong>in</strong>g these stakeholders to comment on<br />

drafts before review.<br />

Word choice: The language used by a project can itself be contentious. Us<strong>in</strong>g appropriate<br />

language (e.g. ‘context analysis’ <strong>in</strong>stead <strong>of</strong> conflict analysis, ‘tension’ <strong>in</strong>stead <strong>of</strong> conflict) can<br />

help. CAEs are <strong>of</strong>ten characterized by significant donor overlap. Multiple projects may work with<br />

the same partner bus<strong>in</strong>esses and households. Moreover, the results that you are try<strong>in</strong>g to<br />

achieve will be affected by many other actors, <strong>in</strong>clud<strong>in</strong>g government, local communities, and<br />

changes <strong>in</strong> the broader context. In such an environment, recognis<strong>in</strong>g the contributions <strong>of</strong> these<br />

other actors is essential.<br />

8 Manag<strong>in</strong>g the System for Results Measurement<br />

A successful results measurement system must be <strong>in</strong>tegrated <strong>in</strong>to all aspects <strong>of</strong> programme<br />

management, from design through implementation to monitor<strong>in</strong>g and evaluation. This section suggests<br />

ways <strong>in</strong> which the management system can be adjusted to CAEs.<br />

8.1 Establish<strong>in</strong>g a clear system for results measurement (Control Po<strong>in</strong>ts 8.1)<br />

Results measurement systems should use the follow<strong>in</strong>g approach to ease adoption.<br />

Invest from the beg<strong>in</strong>n<strong>in</strong>g. Many results measurement systems are added to exist<strong>in</strong>g projects after<br />

their launch, rather than be<strong>in</strong>g designed from <strong>in</strong>ception. This <strong>of</strong>ten means it is too late to generate<br />

quality data – for example, because basel<strong>in</strong>e data is not collected – and creates additional<br />

challenges for staff and partners who must adapt to a new system. It is worth <strong>in</strong>vest<strong>in</strong>g additional<br />

resources dur<strong>in</strong>g the design stage to ensure that the results measurement system is robust from the<br />

beg<strong>in</strong>n<strong>in</strong>g.<br />

Phase <strong>in</strong> the monitor<strong>in</strong>g system. The <strong>DCED</strong> Standard can be demand<strong>in</strong>g, especially for new<br />

partners. Consider phas<strong>in</strong>g <strong>in</strong> the results measurement system, successfully implement<strong>in</strong>g one<br />

component before mov<strong>in</strong>g on to the next. This <strong>in</strong>creases the likelihood <strong>of</strong> the results measurement<br />

system be<strong>in</strong>g used for project management. For example, you may <strong>in</strong>itially require new partners to<br />

monitor and report only aga<strong>in</strong>st activities. Once that is established, move on to monitor<strong>in</strong>g outputs,<br />

then outcomes. At each stage, give further tra<strong>in</strong><strong>in</strong>g, feedback and encouragement. Ensure that you<br />

have matched the complexity <strong>of</strong> the system with the ability <strong>of</strong> your partners, and have a clear plan<br />

for mov<strong>in</strong>g towards implementation <strong>of</strong> the full Standard as soon as possible. However, make sure<br />

66 MEDA, Monday <strong>Development</strong> Magaz<strong>in</strong>e, July 2010, 14.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

38


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

that partners are aware <strong>of</strong> this plan, so that they understand and can prepare for the change <strong>in</strong> the<br />

system.<br />

Require more regular report<strong>in</strong>g. Shorter report<strong>in</strong>g periods may be appropriate <strong>in</strong> more difficult<br />

environments. A DFID-funded programme <strong>in</strong> the DRC changed from six monthly to quarterly<br />

progress report<strong>in</strong>g. It also adopted monthly targets to support a more flexible management<br />

approach <strong>in</strong> the early stages <strong>of</strong> the project. 67 This is particularly effective if it is tied to opportunities<br />

for the project to revise its results cha<strong>in</strong>s and <strong>in</strong>dicators if necessary, and based on a realistic<br />

assessment <strong>of</strong> what the programme can provide.<br />

Knowledge management. Given the frequent high turnover <strong>of</strong> staff, it is essential to properly<br />

document the results management system. This should <strong>in</strong>clude a clear explanation <strong>of</strong> the results<br />

cha<strong>in</strong>, monitor<strong>in</strong>g plan, and systems for collect<strong>in</strong>g and analys<strong>in</strong>g results. It should also cover key<br />

decisions and compromises that were made. For example, if the results cha<strong>in</strong> was simplified to<br />

improve usability, document the changes that were made and the reasons beh<strong>in</strong>d them. This will<br />

prevent future managers from redo<strong>in</strong>g the same work.<br />

8.2 Human and f<strong>in</strong>ancial resources (Control Po<strong>in</strong>t 8.2)<br />

Recruit<strong>in</strong>g and reta<strong>in</strong><strong>in</strong>g qualified staff is a serious challenge <strong>in</strong> CAEs where local capacity may be low,<br />

<strong>in</strong>ternational staff unwill<strong>in</strong>g to work and turnover extremely high.<br />

Consider the composition <strong>of</strong> your team. Are they all from the same region or ethnic group<br />

International organisations commonly hire staff with a m<strong>in</strong>imum level <strong>of</strong> education – who may<br />

primarily be from historically dom<strong>in</strong>ant groups. This may reduce the perceived neutrality <strong>of</strong> your<br />

<strong>in</strong>tervention, or impede cooperation from local communities and government.<br />

Ensure that the team are well <strong>in</strong>formed about conflict, and conflict sensitivity. Try to hire staff with<br />

experience work<strong>in</strong>g <strong>in</strong> conflicts, and deliver additional tra<strong>in</strong><strong>in</strong>g on conflict sensitivity as necessary.<br />

Staff may be <strong>in</strong>itially reluctant to report on conflict, and may need additional tra<strong>in</strong><strong>in</strong>g to encourage<br />

them to do so. It may also be necessary to hire staff focused specifically on conflict sensitivity.<br />

Invest <strong>in</strong> tra<strong>in</strong><strong>in</strong>g courses, mentor<strong>in</strong>g and additional support for local staff. Give regular tra<strong>in</strong><strong>in</strong>g,<br />

opportunities for staff to share lessons, and constructive feedback on the monitor<strong>in</strong>g <strong>in</strong>formation<br />

that they provide.<br />

Tra<strong>in</strong> local partners or communities to assume additional M&E responsibilities. The Aga Khan<br />

<strong>Development</strong> Network, for example, has supported Social Audit Committees, elected by the village<br />

to scrut<strong>in</strong>ize the village council accounts. Aga Khan tra<strong>in</strong>s the committees on basic book-keep<strong>in</strong>g,<br />

supports them to exam<strong>in</strong>e the village council accounts, and then holds open village-wide assemblies<br />

where they present their f<strong>in</strong>d<strong>in</strong>gs. 68<br />

Decentralize your monitor<strong>in</strong>g. Staff that are close to the ground are <strong>of</strong>ten better able to adjust <strong>in</strong><br />

conflict situations and also typically less expensive than adopt<strong>in</strong>g a centralized monitor<strong>in</strong>g strategy.<br />

67 Cox, Marcus and Nigel Thornton, Manag<strong>in</strong>g Results <strong>in</strong> CAEs: A stock-take <strong>of</strong> lessons, experience and practice,<br />

2010, 20.<br />

68 C<strong>of</strong>fey International <strong>Development</strong>, The Cost <strong>of</strong> a Good Deed - Monitor<strong>in</strong>g <strong>in</strong> Post-Conflict Environments and the<br />

Application <strong>of</strong> the Logframe, 2009.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

39


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Hire regional consultants. They are generally less expensive than <strong>in</strong>ternational consultants, may<br />

speak the language, have local knowledge, and may be more acceptable to the local community.<br />

This strategy depends on the availability <strong>of</strong> regional experts and thus will be more applicable <strong>in</strong><br />

some contexts than others.<br />

Case Study: Build<strong>in</strong>g Local Capacity <strong>in</strong> Results Measurement <strong>in</strong> Afghanistan 69<br />

The International Fund for Agricultural <strong>Development</strong> (IFAD) Afghanistan builds local capacity<br />

through the Young Pr<strong>of</strong>essionals Programme. They hire recent agriculture sciences graduates, who<br />

have a strong academic background but little work experience. These young pr<strong>of</strong>essionals work for a<br />

year with IFAD’s local partners, conduct<strong>in</strong>g field surveys and site visits <strong>in</strong> order to <strong>in</strong>dependently<br />

verify outputs and results.<br />

The programme views this strategy as creat<strong>in</strong>g multiple benefits. It is able to cheaply employ<br />

motivated, energetic young pr<strong>of</strong>essionals, enabl<strong>in</strong>g effective monitor<strong>in</strong>g <strong>of</strong> remote projects. In<br />

return, the young pr<strong>of</strong>essionals receive a stipend and ga<strong>in</strong> <strong>in</strong>valuable work experience. Follow<strong>in</strong>g the<br />

pilot, 80% <strong>of</strong> participat<strong>in</strong>g graduates found a job <strong>in</strong> their desired fields.<br />

Literature typically suggests devot<strong>in</strong>g 5-10% <strong>of</strong> a project’s budget to results measurement, which is likely<br />

to be more costly <strong>in</strong> CAEs. 70 In particular, there is likely to be a greater need for primary data collection,<br />

due to non-existent or poor quality secondary data. Changes <strong>in</strong> results cha<strong>in</strong>s and <strong>in</strong>dicators created by<br />

shifts <strong>in</strong> the environment will require additional data collection to re-establish basel<strong>in</strong>es. Staff, transport<br />

and accommodation are also frequently more expensive.<br />

In environments with multiple donors, look for opportunities to share results measurement costs. For<br />

example, multi-donor evaluations may be able to attribute change more effectively. They br<strong>in</strong>g together<br />

different viewpo<strong>in</strong>ts, and reduce the cost both to the commission<strong>in</strong>g partner and the recipient<br />

stakeholders. If it is impossible to dist<strong>in</strong>guish between the effects <strong>of</strong> different donor efforts, it may be<br />

more realistic to evaluate the donor effort as a whole. However, this can also br<strong>in</strong>g additional logistical<br />

and management challenges, as it may require coord<strong>in</strong>ation between donors with different political<br />

agendas and theories <strong>of</strong> change. 71<br />

8.3 Key resources<br />

The People <strong>in</strong> Aid Code is an <strong>in</strong>ternationally recognised management tool that aims to help<br />

humanitarian and development organisations improve the quality <strong>of</strong> their human resources<br />

management.<br />

69 Interview with Abdul Latif Zahed, IFAD Afghanistan, September 17, 2012.<br />

70 Church and Roberts. Design<strong>in</strong>g for Results, Integrat<strong>in</strong>g Monitor<strong>in</strong>g and Evaluation <strong>in</strong>to Conflict Transformation<br />

Programmes, 2006, 132; DFID. <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict Affected States and<br />

Situation, 18; Dalberg, Adapt<strong>in</strong>g IFC M&E approaches for projects <strong>in</strong> conflict affected countries, Phase 1: Formative<br />

research and key issue diagnosis, 7.<br />

71 OECD, Encourag<strong>in</strong>g Effective Evaluation <strong>of</strong> Conflict Prevention and Peacebuild<strong>in</strong>g Activities, 2007.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

40


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Annex 1: Key Def<strong>in</strong>itions 72<br />

Activity: A discrete piece <strong>of</strong> work, typically represented by a contract between the programme and a<br />

contractor, partner or consultant. Interventions typically consist <strong>of</strong> several activities, that are <strong>in</strong>tended<br />

to achieve change at various different po<strong>in</strong>ts <strong>in</strong> the overall market system.<br />

Assumption: A supposition or best guess which forms part <strong>of</strong> the basis for calculation <strong>of</strong> an <strong>in</strong>dicator<br />

value.<br />

Attribution: The ascription <strong>of</strong> a causal l<strong>in</strong>k between observed (or expected to be observed) changes and<br />

a specific <strong>in</strong>tervention<br />

Basel<strong>in</strong>e: An analysis describ<strong>in</strong>g the situation prior to a development <strong>in</strong>tervention, aga<strong>in</strong>st which<br />

progress can be assessed or comparisons made.<br />

Conflict: The result <strong>of</strong> parties disagree<strong>in</strong>g (e.g. about the distribution <strong>of</strong> material or symbolic resources)<br />

and act<strong>in</strong>g on the basis <strong>of</strong> these perceived <strong>in</strong>compatibilities. 73<br />

Conflict affected environments: Environments that have recently been, currently are or are prone to<br />

be<strong>in</strong>g affected by conflict.<br />

Conflict analysis: The systematic study <strong>of</strong> the pr<strong>of</strong>ile, causes, actors, and dynamics <strong>of</strong> conflict. 74<br />

Conflict sensitivity: The ability <strong>of</strong> an organisation to understand the context <strong>in</strong> which it operates;<br />

understand the <strong>in</strong>teraction between an <strong>in</strong>tervention and the context; and act upon the understand<strong>in</strong>g <strong>of</strong><br />

this <strong>in</strong>teraction, <strong>in</strong> order to avoid negative impacts and maximise positive impacts. 75<br />

Copy<strong>in</strong>g: Other target enterprises copy<strong>in</strong>g behavioural changes that those affected directly by<br />

programme activities have adopted. 76<br />

Counterfactual: The situation or condition which hypothetically may prevail for <strong>in</strong>dividuals,<br />

organizations, or groups were there no development <strong>in</strong>tervention.<br />

Crowd<strong>in</strong>g <strong>in</strong>: Enterprises at levels other than the target level copy<strong>in</strong>g behaviours that those affected by<br />

programme activities have adopted or enter<strong>in</strong>g a sector or value cha<strong>in</strong> as a result <strong>of</strong> improved <strong>in</strong>centives<br />

and environment created (at least partly) by the programme. This term also applies to government<br />

agencies or civil society organizations, who are not directly <strong>in</strong>volved <strong>in</strong> the programme, copy<strong>in</strong>g<br />

72 Unless noted, all def<strong>in</strong>itions drawn from OECD, Glossary <strong>of</strong> Key Terms <strong>in</strong> Evaluation and Results Based<br />

Management, 2002.<br />

73 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack, 3.<br />

74 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack,<br />

chapter 2, page 1<br />

75 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack, 3.<br />

76 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />

<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 15.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

41


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

behaviours <strong>of</strong> those who are directly <strong>in</strong>volved <strong>in</strong> the programme, or who change their behaviour as a<br />

result <strong>of</strong> improved <strong>in</strong>centives or environment created (at least partly) by the programme.. 77<br />

Displacement: Some enterprises may be negatively affected because others are benefit<strong>in</strong>g from<br />

programme activities. Displacement is the amount <strong>of</strong> negative effect on those enterprises harmed by<br />

programme activities. 78<br />

Do No Harm: Ways <strong>in</strong> which <strong>in</strong>ternational humanitarian and development assistance given <strong>in</strong> conflict<br />

sett<strong>in</strong>gs may be provided so that, rather than exacerbat<strong>in</strong>g and worsen<strong>in</strong>g the conflict, it helps local<br />

people disengage from fight<strong>in</strong>g and develop systems for settl<strong>in</strong>g the problems which prompt conflict<br />

with<strong>in</strong> their societies.<br />

Evaluation: The systematic and objective assessment <strong>of</strong> an on-go<strong>in</strong>g or completed project, programme<br />

or policy, its design, implementation and results. Evaluation also refers to the process <strong>of</strong> determ<strong>in</strong><strong>in</strong>g<br />

the worth or significance <strong>of</strong> an activity, policy or program. An assessment, as systematic and objective as<br />

possible, <strong>of</strong> a planned, on-go<strong>in</strong>g, or completed development <strong>in</strong>tervention.<br />

Impacts: Positive and negative, primary and secondary long-term effects produced by a development<br />

<strong>in</strong>tervention, directly or <strong>in</strong>directly, <strong>in</strong>tended or un<strong>in</strong>tended.<br />

Intervention: A coherent set <strong>of</strong> activities that share a s<strong>in</strong>gle results cha<strong>in</strong>, and are designed to achieve a<br />

specific and limited change. 79 A project usually manages multiple <strong>in</strong>terventions and thus multiple results<br />

cha<strong>in</strong>s.<br />

Monitor<strong>in</strong>g: A cont<strong>in</strong>u<strong>in</strong>g function that uses systematic collection <strong>of</strong> data on specified <strong>in</strong>dicators to<br />

provide management and the ma<strong>in</strong> stakeholders <strong>of</strong> an ongo<strong>in</strong>g development <strong>in</strong>tervention with<br />

<strong>in</strong>dications <strong>of</strong> the extent <strong>of</strong> progress and achievement <strong>of</strong> objectives and progress <strong>in</strong> the use <strong>of</strong> allocated<br />

funds.<br />

Peacebuild<strong>in</strong>g: Measures designed to consolidate peaceful relations and strengthen viable political,<br />

socio-economic and cultural <strong>in</strong>stitutions capable <strong>of</strong> mediat<strong>in</strong>g conflict, and to strengthen other<br />

mechanisms that will either create or support the necessary conditions for susta<strong>in</strong>ed peace. 80<br />

Proxy <strong>in</strong>dicator: An <strong>in</strong>dicator for which measurable change is clearly and reliably correlated with an<br />

<strong>in</strong>dicator <strong>of</strong> a change that the programme aims to achieve (but is generally more practical to measure). 81<br />

77 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />

<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 15.<br />

78 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />

<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 16.<br />

79 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />

<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 16.<br />

80 Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource Pack, 5.<br />

81 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />

<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 17.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

42


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Results cha<strong>in</strong>: The causal sequence for a development <strong>in</strong>tervention that stipulates the necessary<br />

sequence to achieve desired objectives beg<strong>in</strong>n<strong>in</strong>g with <strong>in</strong>puts, mov<strong>in</strong>g through activities and outputs,<br />

and culm<strong>in</strong>at<strong>in</strong>g <strong>in</strong> outcomes, impacts and feedback.<br />

Results measurement: The process <strong>of</strong> design<strong>in</strong>g a measurement system <strong>in</strong> order to estimate a<br />

programme’s impact so that it can be used to report results and improve project management. 82<br />

Susta<strong>in</strong>ability: The cont<strong>in</strong>uation <strong>of</strong> benefits from a development <strong>in</strong>tervention after major development<br />

assistance has been completed. The probability <strong>of</strong> cont<strong>in</strong>ued long term benefits.<br />

Systemic change: Change <strong>in</strong> systems that are caused by <strong>in</strong>troduc<strong>in</strong>g alternative <strong>in</strong>novative susta<strong>in</strong>able<br />

bus<strong>in</strong>ess models at support market level (such as <strong>in</strong> private sector, government, civil society, public<br />

policy level). These changes <strong>of</strong>ten cause widespread <strong>in</strong>direct impact by crowd<strong>in</strong>g <strong>in</strong> at support market<br />

levels impact and copy<strong>in</strong>g at f<strong>in</strong>al beneficiary level. 83<br />

Theory <strong>of</strong> Change: A testable hypothesis regard<strong>in</strong>g how the planned activities will contribute to<br />

achiev<strong>in</strong>g the desired results for the programme. 84<br />

Triangulation: The use <strong>of</strong> multiple theories, sources or types <strong>of</strong> <strong>in</strong>formation, or types <strong>of</strong> analysis to<br />

verify and substantiate an assessment.<br />

82 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />

<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 17.<br />

83 The Donor Committee for Enterprise <strong>Development</strong>, The <strong>DCED</strong> Standard for <strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>in</strong> <strong>Private</strong><br />

<strong>Sector</strong> <strong>Development</strong>: Control Po<strong>in</strong>ts and Compliance Criteria, Version VI, 2013, 18.<br />

84 CARE, International UK, Peacebuild<strong>in</strong>g with Impact: Def<strong>in</strong><strong>in</strong>g Theories <strong>of</strong> Change, 2012, 3.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

43


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Annex 2: Acronyms and Abbreviations<br />

BMO Bus<strong>in</strong>ess Membership Organization<br />

CAE<br />

<strong>DCED</strong><br />

DFID<br />

IFAD<br />

IFC<br />

IRC<br />

MSA<br />

M&E<br />

NGO<br />

OECD<br />

RCT<br />

Conflict Affected Environment<br />

Donor Committee for Enterprise <strong>Development</strong><br />

Department for International <strong>Development</strong><br />

International Fund for Agricultural <strong>Development</strong><br />

International F<strong>in</strong>ance Corporation<br />

International Research Committee<br />

MarketShare Associates<br />

Monitor<strong>in</strong>g and Evaluation<br />

Non-Governmental Organization<br />

Organisation for Economic Co-operation and <strong>Development</strong><br />

Randomized Control Trial<br />

UNDP United Nations <strong>Development</strong> Programme<br />

USAID United States Agency for International <strong>Development</strong><br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

44


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Annex 3:<br />

Overall Resource List<br />

Onl<strong>in</strong>e Resources<br />

<strong>DCED</strong> <strong>Measur<strong>in</strong>g</strong> Results Resources. Provides several key resources for results measur<strong>in</strong>g <strong>of</strong> PSD and<br />

peacebuild<strong>in</strong>g.<br />

<strong>DCED</strong> Onl<strong>in</strong>e Library on PSD <strong>in</strong> Conflict-Affected Environments. Provides access to more than 450<br />

documents on the theme.<br />

<strong>DCED</strong> Compilation <strong>of</strong> Guidance by Various Agencies for Practitioners <strong>of</strong> PSD <strong>in</strong> Conflict-Affected<br />

Environments. L<strong>in</strong>ks to exist<strong>in</strong>g advice and guidance for assessment, programme design and<br />

implementation and measur<strong>in</strong>g results.<br />

Design, Monitor<strong>in</strong>g and Evaluation for Peacebuild<strong>in</strong>g. Offers resources and discussions on resources<br />

for monitor<strong>in</strong>g and evaluation <strong>of</strong> peacebuild<strong>in</strong>g programm<strong>in</strong>g.<br />

Organization for Economic Co-operation and <strong>Development</strong>. Overview <strong>of</strong> the Network's Current Work<br />

On Evaluat<strong>in</strong>g Conflict Prevention And Peacebuild<strong>in</strong>g. Presents the Organization for Economic Cooperation<br />

and <strong>Development</strong> (OECD)’s conflict prevention and peacebuild<strong>in</strong>g evaluations.<br />

Conflict Analysis Guidel<strong>in</strong>es<br />

Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian Assistance and Peace Build<strong>in</strong>g Resource<br />

Pack, Chapter Two. Very comprehensive conflict sensitivity guidel<strong>in</strong>es. Chapter two describes conflict<br />

analysis <strong>in</strong> detail. Of particular <strong>in</strong>terest are pages 12-40 <strong>in</strong> chapter two, which summarise 15 different<br />

conflict analysis tools from major donors. For each they discuss the primary purpose, summarise the<br />

ma<strong>in</strong> steps, and give examples <strong>of</strong> lessons learned and current applications. If you are select<strong>in</strong>g a conflict<br />

analysis framework, this is an <strong>in</strong>valuable resource.<br />

Collaborative for <strong>Development</strong> Action, Inc. The Do No Harm Handbook: The Framework for Analyz<strong>in</strong>g<br />

the Impact <strong>of</strong> Assistance on Conflict. This guide provides a step-by-step guide to assess<strong>in</strong>g the potential<br />

<strong>of</strong> a project to cause harm <strong>in</strong> its selected context.<br />

GIZ. Peace and Conflict Assessment (PCA): A Methodological Framework for the Conflict- and Peace-<br />

Oriented Alignment <strong>of</strong> <strong>Development</strong> Programmes. Exam<strong>in</strong>es the application <strong>of</strong> peace and conflict<br />

assessments at multiple stages <strong>in</strong> a project, <strong>in</strong>clud<strong>in</strong>g impact monitor<strong>in</strong>g.<br />

USAID. Conflict Assessment Framework. Version 2.0. Provides a framework for diagnos<strong>in</strong>g a conflict<br />

and generat<strong>in</strong>g responses. Application guidel<strong>in</strong>es can be found here.<br />

Results Cha<strong>in</strong>s and Theories <strong>of</strong> Change <strong>in</strong> CAEs<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

45


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Becker, Sab<strong>in</strong>e. Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes. Reviews<br />

many <strong>of</strong> the key relationships between PSD and peacebuild<strong>in</strong>g.<br />

CARE International UK. Peacebuild<strong>in</strong>g with Impact: Def<strong>in</strong><strong>in</strong>g Theories <strong>of</strong> Change. Offers guidance <strong>in</strong><br />

develop<strong>in</strong>g the logic <strong>of</strong> how project <strong>in</strong>terventions will impact upon peacebuild<strong>in</strong>g.<br />

Davis, Peter. Corporations, Global Governance and Post-Conflict Reconstruction. Synthesizes the<br />

impacts corporations may have on a conflict and support<strong>in</strong>g reconstruction, draw<strong>in</strong>g from Azerbaijan,<br />

Bosnia and Rwanda.<br />

Humphreys, Macartan. Economics and Violent Conflict. Reviews the evidence for l<strong>in</strong>kages between<br />

economics and conflict.<br />

Mercy Corps. Peacebuild<strong>in</strong>g through Economic <strong>Development</strong> Approach. This publication outl<strong>in</strong>es five<br />

theories <strong>of</strong> change that Mercy Corps is implement<strong>in</strong>g and test<strong>in</strong>g through its work and provides<br />

examples <strong>of</strong> each.<br />

Mierke, Axel. Conflict Prevention and Peace Build<strong>in</strong>g Elements <strong>of</strong> PSD/SED Programmes. Analyzes the<br />

potential support <strong>of</strong> private sector development programm<strong>in</strong>g to peacebuild<strong>in</strong>g.<br />

Pottebaum, David. Relationships between Conflict, Poverty, Inequality, and Economic Growth. This<br />

publication reviews the evidence for relationships between conflict, poverty, <strong>in</strong>equality, and economic<br />

growth.<br />

Sen, Nabanita. A Guide to Mak<strong>in</strong>g Results Cha<strong>in</strong>s. Provides the key steps required to create a results<br />

cha<strong>in</strong>.<br />

USAID. Conflict-Sensitive Approaches to Value Cha<strong>in</strong> <strong>Development</strong>, microNote #101. Outl<strong>in</strong>es how to<br />

<strong>in</strong>corporate a conflict lens when conduct<strong>in</strong>g research on value cha<strong>in</strong>s that can <strong>in</strong>form the results cha<strong>in</strong><br />

logic.<br />

Vogel, Isabel. Review <strong>of</strong> the uses <strong>of</strong> ‘Theory <strong>of</strong> Change’ <strong>in</strong> International <strong>Development</strong>. This report<br />

exam<strong>in</strong>es how theories <strong>of</strong> change are used <strong>in</strong> the field, different applications and best practice.<br />

Conflict Sensitivity<br />

Conflict Sensitivity Consortium. Conflict-Sensitive Approaches to <strong>Development</strong>, Humanitarian<br />

Assistance and Peace Build<strong>in</strong>g Resource Pack. Provides guidance on how to <strong>in</strong>corporate a conflict<br />

sensitive approach throughout the project cycle, <strong>in</strong>clud<strong>in</strong>g monitor<strong>in</strong>g and evaluation.<br />

Conflict Sensitivity Consortium. A How To Guide to Conflict Sensitivity. Based on the above<br />

document, this adopts a similar approach. It is updated and shortened slightly from the orig<strong>in</strong>al resource<br />

pack.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

46


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

Def<strong>in</strong><strong>in</strong>g Indicators<br />

Agoglia et al. <strong>Measur<strong>in</strong>g</strong> Progress <strong>in</strong> Conflict Environments. Provides a significant volume <strong>of</strong> outcome<br />

<strong>in</strong>dicators around five aspects <strong>of</strong> a conflict, <strong>in</strong>clud<strong>in</strong>g the existence <strong>of</strong> a susta<strong>in</strong>able economy. These are<br />

high-level <strong>in</strong>dicators for gaug<strong>in</strong>g the overall stability <strong>of</strong> a situation, rather than the direct success <strong>of</strong> an<br />

<strong>in</strong>dividual project<br />

CDA. Indications for Assess<strong>in</strong>g Aid’s Impact on Conflict. Provides various <strong>in</strong>dications <strong>of</strong> a project’s<br />

positive and negative effects on the conflict.<br />

Church, Cheyanne and Mark M. Roberts. Design<strong>in</strong>g for Results: Integrat<strong>in</strong>g M&E <strong>in</strong> Conflict<br />

Transformation Programmes. General guidel<strong>in</strong>es on monitor<strong>in</strong>g and evaluat<strong>in</strong>g <strong>in</strong> CAEs. Chapter four<br />

<strong>in</strong>cludes a good discussion on <strong>in</strong>dicators and some helpful examples.<br />

DFID. Interim Guidance Note: <strong>Measur<strong>in</strong>g</strong> and Manag<strong>in</strong>g for Results <strong>in</strong> Fragile and Conflict-Affected<br />

States and Situations. Annex A lists <strong>of</strong> <strong>in</strong>dicators for high level objectives, and possible data sources.<br />

More focused on country programmes than projects.<br />

Mercy Corps. Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions. Conflict & Economics:<br />

Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Provides a menu <strong>of</strong> 29 peace and conflict <strong>in</strong>dicators and 49<br />

economics and conflict <strong>in</strong>dicators. Each <strong>in</strong>dicator is def<strong>in</strong>ed and a list <strong>of</strong> potential disaggregations is<br />

suggested. There is a useful discussion <strong>of</strong> how the <strong>in</strong>dicators were selected and measured, <strong>in</strong>clud<strong>in</strong>g the<br />

tools and surveys that were used.<br />

Indicator Measurement <strong>in</strong> CAEs<br />

Mercy Corps. Evaluation and Assessment <strong>of</strong> Poverty and Conflict Interventions: Conflict & Economics:<br />

Lessons Learned on <strong>Measur<strong>in</strong>g</strong> Impact. Provides several conflict monitor<strong>in</strong>g tools, <strong>in</strong>clud<strong>in</strong>g a violent<br />

<strong>in</strong>cident report<strong>in</strong>g form, a disputes and dispute resolution assessment, a scored community relationship<br />

mapp<strong>in</strong>g tool, and a conflict and resource mapp<strong>in</strong>g tool.<br />

Catley, Burns, Abebe and Suji, Participatory Impact Assessment: A Guide for Practitioners, 2008,<br />

Fe<strong>in</strong>ste<strong>in</strong> Centre. Outl<strong>in</strong>es an eight-step approach to conduct<strong>in</strong>g a participatory impact assessment.<br />

Frontl<strong>in</strong>e SMS. Frontl<strong>in</strong>e SMS is a free s<strong>of</strong>tware for creat<strong>in</strong>g and manag<strong>in</strong>g SMS activities such as<br />

mak<strong>in</strong>g announcements, conduct<strong>in</strong>g polls, and automatically reply<strong>in</strong>g to <strong>in</strong>com<strong>in</strong>g SMS. In particular, it is<br />

excellent for manag<strong>in</strong>g large mobile phone surveys, and was used <strong>in</strong> the above example by Colombia<br />

University to gather data <strong>in</strong> Eastern DRC.<br />

Muaz Jalil. Practical Guidel<strong>in</strong>es for Conduct<strong>in</strong>g Research. Provides a review <strong>of</strong> good practices <strong>in</strong><br />

design<strong>in</strong>g and conduct<strong>in</strong>g research <strong>in</strong> l<strong>in</strong>e with the <strong>DCED</strong> Standard.<br />

Sartorius, Rolf and Christopher Carver. Monitor<strong>in</strong>g, Evaluation and Learn<strong>in</strong>g for Fragile States and<br />

Peacebuild<strong>in</strong>g Programs: Practical Tools for Improv<strong>in</strong>g Program Performance and Results. This<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

47


<strong>Measur<strong>in</strong>g</strong> <strong>Achievements</strong> <strong>of</strong> <strong>Private</strong> <strong>Sector</strong> <strong>Development</strong> <strong>in</strong> Conflict-Affected Environments<br />

provides a large number <strong>of</strong> tools for measur<strong>in</strong>g peacebuild<strong>in</strong>g programmes, alongside tips on<br />

measurement and example <strong>in</strong>dicators.<br />

Attribution <strong>in</strong> CAEs<br />

Evaluation Journal. Special Issue: Contribution analysis. Multiple articles <strong>in</strong>troduc<strong>in</strong>g the concept <strong>of</strong><br />

contribution analysis and its application to results measurement.<br />

Mayne, John. Contribution Analysis: An approach to explor<strong>in</strong>g cause and effect, 2008. A short<br />

<strong>in</strong>troduction to Contribution Analysis.<br />

OECD. Evaluat<strong>in</strong>g Peacebuild<strong>in</strong>g Activities <strong>in</strong> Sett<strong>in</strong>gs <strong>of</strong> Conflict and Fragility: Improv<strong>in</strong>g Learn<strong>in</strong>g for<br />

Results. Provides guidance on assess<strong>in</strong>g attribution <strong>in</strong> the evaluation <strong>of</strong> peacebuild<strong>in</strong>g activities.<br />

Sen, Nabanita. Attribution: <strong>Measur<strong>in</strong>g</strong> Attributable Change Caused by a Programme. A brief,<br />

accessible outl<strong>in</strong>e <strong>of</strong> how to estimate attribution when apply<strong>in</strong>g the <strong>DCED</strong> Standard. A brief, accessible<br />

outl<strong>in</strong>e <strong>of</strong> how to estimate attribution when apply<strong>in</strong>g the <strong>DCED</strong> Standard.<br />

Stern, Elliot et al. Broaden<strong>in</strong>g the Range <strong>of</strong> Designs and Methods for Impact Evaluations. Addresses<br />

the issue <strong>of</strong> attribution <strong>in</strong> depth, look<strong>in</strong>g at the role <strong>of</strong> mixed methods and theory-based approaches to<br />

assess<strong>in</strong>g attribution.<br />

System Management <strong>in</strong> CAEs<br />

The People <strong>in</strong> Aid Code is an <strong>in</strong>ternationally recognised management tool that aims to help<br />

humanitarian and development organisations improve the quality <strong>of</strong> their human resources<br />

management.<br />

Evaluation<br />

OECD. Evaluat<strong>in</strong>g Peacebuild<strong>in</strong>g Activities <strong>in</strong> Sett<strong>in</strong>gs <strong>of</strong> Conflict and Fragility: Improv<strong>in</strong>g Learn<strong>in</strong>g for<br />

Results. Provides guidance on how to conduct evaluations <strong>in</strong> CAEs.<br />

Practical Guidel<strong>in</strong>es for Implement<strong>in</strong>g the <strong>DCED</strong> Standard. Version 2, July 2013<br />

48

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!