Measuring Achievements of Private Sector Development in ... - DCED

Measuring Achievements of Private Sector Development in Conflict-Affected Environments
However, baselines in CAEs can be particularly complex due to the challenges listed above. Moreover,
projects in CAEs are more likely to require a flexible implementation strategy that involves changing
their activities, outputs and outcomes as the project goes on. This could conceivably render original
baseline information irrelevant. The DCED Standard emphasizes the importance of articulating specific
results chains and indicators for each project intervention. Consequently, instead of conducting a single,
large project-wide baseline, each intervention may have its own smaller baseline. This enables projects
to avoid discarding all baseline data if, for instance, violent upheaval renders some of a project’s
interventions unfeasible.
Retrospective baselines are baselines conducted after the intervention has already started, by asking
participants to recall the situation before the start of the project. They may be the only option to collect
information about the past, although their use is not ideal and can produce biased and erroneous
results. 52 If using retrospective baselines, it is important to carefully consider both the time lag since
the event in question has occurred and the type of event that you are asking about. The type of
experience determines how well it can be recalled. For example, questions about actions performed
directly by the respondent are more likely to be recalled. Information that was heard or read – for
example, market access information provided by a service provider in a phone call – will be forgotten
more quickly. A rough guide is to not ask recall questions about events that occurred over a year ago, as
memory becomes increasingly unreliable.
3.2 Good research practices (Control Point 3.2)
While the standard good research practices that are listed in the DCED’s Guide to Conducting Research 53
apply to CAEs, there are other important considerations. The challenges of working in CAEs mean that
the acceptable standard for research practices, including sample sizes and attribution measurement,
(see the section on attribution, below) may be less rigorous. Less exacting standards of practice are
sometimes all that is possible.
This section presents good practices for designing research tools and collection information in CAEs.
Ultimately, maintaining a focus on the three principles for success (simplicity, flexibility, sensitivity) is
key.
Some tips for selecting questions and information tools are:
Use short, simple questions. Simple questions are more easily understood by staff and users
and therefore provide better results. Overly complicated questions can greatly increase the
workload both in data collection and analysis. Similarly, close-ended survey questions are often
preferable for their greater ease of coding and analysing. Focus group discussions and surveys
also should be of a manageable size. One practitioner suggests that 40 survey questions and 10
focus group discussion questions should be the maximum in order to not overwhelm data
52 Ravaillon, Martin, Can We Trust Shoestring Evaluations Policy Research Working Paper 5983, 2012.
53 Donor Committee for Enterprise Development, Measuring Achievements in Private Sector Development
Implementation Guidelines, Version 1g, 5 th March 2010, 26-27.
Practical Guidelines for Implementing the DCED Standard. Version 2, July 2013
29