Download Full Report (PDF, 1.36MB) - Queen's University Belfast

2.3 The influence of implementation quality on effectiveness
Chapter 2: Literature Review
The effectiveness of pro-social behaviour programmes depends on their implementation quality (Derzon et al, 2005;
Durlak and DuPre, 2008; Cross et al, 2010; Farrell et al, 2001; Gottfredson and Gottfredson, 2002; Dusenbury et al, 2005).
In a meta-analytic study, Derzon et al (2005) showed that only substance abuse interventions with high implementation
quality exhibited a significant effect. Durlak and DuPre (2008) showed that the outcomes of over 80 school-based
prevention programmes were influenced by implementation quality. Furthermore, Cross et al (2010) showed that a
number of contextual factors, such as communities’ delivery and support systems, had an influence on effectiveness.
Losel and Beelman (2003) showed instructional modality to be a significant predictor of the development of pro-social
skills among the studies that reported it. Active instructional modalities (i.e. group work such as role plays) demonstrated
a significantly greater effect size than passive, traditional instructional modalities (i.e. presentations). Also, the amount of
exposure to the intervention was a significant predictor of effects. A study of the implementation of the Coping Power
Program found that its effectiveness and implementation may depend on the training intensity and quality of those who
deliver it (Lochman et al, 2009).
A significant review by Durlak et al (2011) has recommended that social and emotional learning programme implementation
will result in greater effects if it follows ‘SAFE’ training practices, meaning:
• Sequenced and step-by-step training approach (building up concepts).
• Active forms of learning (like group work and role plays).
• Focused time on skills development (such as emotional intelligence skills).
• Explicit learning goals (demonstrations of appropriate outcomes, e.g., pro-social behaviour).
These SAFE practices have also been found to be an influencing factor in the effectiveness of after-school programmes
when developing personal and social skills (Durlak et al, 2010).
In addition, programme complexity, fidelity, mismatch between school and intervention’s mission, lack of training and
support present another set of implementation issues (Dumas et al, 2001; Hallfors and Godette, 2002; Hallfors and
Van Dorn, 2002; Thaker et al, 2008). For example, Hallfors and Godette (2002) reported that only 19% of evidencebased
prevention programmes reported fidelity, an essential measure of whether an intervention meets its theoretical
goals and whether it is conducted in a consistent way with all participants. Low fidelity would directly have an effect
on the programme’s validity and indirectly affect the study’s power (Dumas et al, 2001). Furthermore, Thaker et al
(2008) described a variety of issues that arose during implementation of their Reconnecting Youth (RY) programme.
They identified: (a) a lack of properly trained staff or inability to train available staff; (b) substantial change within schools
(e.g. repairs, reforms); and (c) a lack of support, either financial or administrative. Overall, the implementation of these
programmes requires a combination of clear leadership, supportive staff, availability of resources and partnerships with
parents, community, pupils and other agencies. It is worth highlighting that all of these factors form part of the Childhood
Development Initiative (CDI) strategy.
Implementation also requires the development of systems, structures and practices whereby the implementation of the
programme can be monitored and reviewed on an ongoing basis (Domitrovich et al, 2008). It should be understood that
implementation takes time to embed and positive changes may not be immediately apparent (Adelman and Taylor, 2003;
Dusenbury and Hansen, 2004).
11