Learning from Nine Examples of Peacebuilding Evaluation

According to Mercy Corps, the use of the mixed-method approach allowed
them not only to credibly claim program impact through the use of surveys
and comparison groups but also to gather descriptive program information
and learn more about the mechanisms of program impact through qualitative
research.
The purpose of the evaluation was two-fold: first, to see if the program
produced the desired set of interdependent outcomes, including increased
security, improved relationships between communities in conflict, and improved
access to resources, and second, to test the program’s basic theory
of change, that increased economic interaction increases stability.
The findings of the evaluation were: 1) security improved across the subregion
over the life of the program; 2) communities where the program was
implemented experienced improved access to resources, increased perceptions
of security, increased trust between adversarial communities, and
strengthened intercommunal ties compared with communities where the
program was not implemented; and 3) the quality of relationships between
communities significantly improved, particularly in terms of trust levels and
perceptions of relationships between the conflicting communities.
Considerations on the Methodology
Strengths
• A strength of the method was the clear articulation of a theory of change.
Chris Blattman recently noted that impact evaluation should, “test ideas,
not programs.” 7 Mercy Corps’ clear and simple articulation of their theory of
change allowed them to test more general ideas about the linkage between
economic relations and conflict as well as their specific program approach.
31
• A second strength of the method was its commitment to a mixed-methods
approach. The use of comparison groups in the surveys allowed for a relatively
strong claim to program impact even as the situation in the region was
improving overall. Alternatively, qualitative research allowed the creation
of more rich, detailed knowledge on a nuanced and complex topic. It also
allowed inquiry into the mechanisms of program impact that could not be
illuminated through the survey work.
Challenges and Pitfalls
• Mercy Corps and Evidence Summit participants discussed several challenges
and potential issues with the evaluation strategy. Several related to the complexity
of the evaluation. According to Mercy Corps, they were overwhelmed
7 Chris Blattman, “Impact Evaluation 3.0,” September 2, 2011,
http://chrisblattman.com/2011/09/02/impact-evaluation-3-0