Campaigns to End Violence against Women and Girls - Virtual ...

Advanced research designs for impact assessment
To establish robust links between campaign exposure, and impact in terms of attitudes
and behaviour-change, the following methods can generate a reliable comparison
between members of the target audience that have been exposed to the campaign and
those who have not been exposed:
Experimental trial is widely considered to be the most accurate way to assess impact,
e.g. of a communication intervention. Popularly known in evaluation circles as RCTs
(random controlled trials) and much used in scientific research, this method identifies
the difference between what a campaign achieved and what would have been achieved
without the campaign.
Example: The evaluation of a media campaign on reducing sexual violence in high
schools could for example use an approach to assess impact by conducting a formal
experimental trial involving a control and an experimental group. The evaluation could
be conducted in two different towns with similar socio-economic conditions. In each high
school, the 11th grade males and females could be administered an initial KAP survey
on the topic (pre-test). After campaign exposure over a predetermined spell of time at
the “experimental” school (the students in the control high school would not be exposed
to the campaign), 11th graders at both schools would be administered the same survey
again (post-test). Comparing the pre- and post-test results at the two schools will allow
for the drawing of certain conclusions on the effectiveness of the campaign – bearing in
mind that other factors not considered in research design may have exerted an influence
as well.
Source: Source: Potter, S. (2008): Incorporating Evaluation into Media Campaign
Design. Harrisburg, PA, on VAWnet, a project of the National Resource Center on
Domestic Violence/Pennsylvania Coalition Against Domestic Violence.
Experimental trials can produce reliable results if they are administered at a significant
scale and with scientific rigor, which requires substantial resources in terms of time,
skills and money. Ethical issues need to be taken into account as well – by definition,
the control group will be excluded from campaign activities and any potential benefits
involved.
There are a series of alternative methods to RCT that are accepted as producing equally
valid results while demanding less time and offering more flexibility:
� Repeated Measures: The same measurement, e.g. the same questionnaire, is
administered to the same individuals or groups at intervals, e.g. every six months, to
verify how their response has evolved since the beginning and at different phases of
the campaign.
� Staged Implementation: If a campaign is rolled out in different phases with a
substantial time lag, the evaluation can compare areas exposed to the campaign in
its early stages to areas that have not yet been exposed.
� Natural Variations in Treatment: In a large-scale campaign, implementation in
some areas is bound to “fail” or not roll out exactly as intended. If these variations
can be adequately tracked and measured, they can provide useful comparisons for
impact assessment.
291
Campaigns December 2011