Learning from Nine Examples of Peacebuilding Evaluation

Key findings of the case studies were: practitioners highly valued the training
and ongoing specialist support; parents welcomed the opportunity for their
children to learn about respecting difference; committed leadership within
the setting where the curriculum was delivered was crucial; and integration
of the various aspects of the project across different settings was important
to successful implementation.
Considerations on the Methodology
Strengths
• While the Evidence Summit was not designed to produce a rigorous peer
review of the evaluation methodology, the RCT discussed here appears to
have been carefully constructed and implemented. The methodology included
true randomization, sufficient power, and credible instruments to measure
the change in attitudes among the children.
• The benefit of this methodology, according to Early Years, was that it produced
a simple, powerful measure of impact, which was important for their
engagement with donors and with skeptics who believed that introducing
these issues to young children would be counterproductive.
• The inclusion of the qualitative research broadened the appeal of the
research and generated crucial buy-in from staff members, parents, and educators.
The results of the qualitative research, Early Years reports, were more
important for stakeholders on a day-to-day basis.
25
Challenges and Pitfalls
• The first key challenge was the financial and other costs of the RCT. Due
to the disruption caused within the schools and the complexity and cost of
the research, the evaluation was necessarily a one-off initiative. It was not a
process that could feasibly be repeated at regular intervals to create ongoing
learning. The organization continues to evaluate their programs but now
primarily uses more qualitative, participatory approaches.
• The use of external researchers created problems. Early Years had to
balance the demands of the research with their own community development
principles and the dynamic nature of the projects being assessed. In
addition, the researchers needed to do their own learning of the work of Early
Years. Finally, Early Years noted that the design limited research capacity
building within the organization.
• Early Years reported that the qualitative research did not capture the richness
of the program, nor the full impact, especially with parents. The bulk of the
resources for the initiative went to the RCT. The qualitative research was not
given equal emphasis, which may have created some of these shortcomings.