Deliverable D 1.3 RESULTS OF META-ANALYSIS ... - cast-eu.org

Campaigns and awareness raising strategies in traffic safety — Deliverable D-1.130-fold increases in percentage of people using seatbelts. Filling in‘missing’ values by generating 30-fold decreases in seatbelt use is notsensible.• Caution should be taken when comparing corrected overall effectsgenerated from unequal sized sets of effects, especially where one groupof effects has n < 15. This is because the effects of detecting andcompensating for publication bias are more extreme for smaller groups ofeffects.Table 7.18 shows overall effects of road safety campaigns on accident countsor seatbelt usage rates, before and after correcting for publication bias.Although the effects remain significant, they are somewhat reduced because26 campaigns have been added to compensate for assumed publication bias.The resulting conservative estimate suggests that road safety campaignsreduce accident numbers by 7% (Table 7.18).Table 7.18 Overall effect of road safety campaigns on accidentsOutcome measureAccount for publication bias?effect(%)95%confidenceintervalSignificant?(p < 0.05)No - 10 (- 11 -10) yesAccidents (n = 115)Yes, generate 26 effects tocompensate- 7 (- 8, -6) yesStep 3 & 4 Fixed- and random-effects: homogeneity and heterogeneityThe derivation of overall effect described in step 1 is based on a fixed-effectsmodel, which accounts only for chance variation in effect, and not anyunknown systematic variation. Overall effects derived using a fixed-effectsmodel predict smaller variations than those derived using a random-effectsmodel, because the latter accounts for unknown systematic variations. Todecide which model to use we look at the amount of variation among a set ofeffects. The higher the variation, the greater the chance of systematic causesof variation, and the stronger the random-effects model is recommended.Variation is examined using a test for homogeneity that outputs a test statisticcalled Cochrane’s Q (Christensen, 2003). Where Q is significantly larger thanexpected for a homogenous data set, homogeneity is rejected, and a randomeffectsmodel employed.For our set of accident effects, Q is significantly larger than expected forhomogenous data set and a random effects model is therefore used todescribe the overall effect. The random effects model derives an overall effectof campaigns on accidents of -13%. As shown in Table 7.19, although theapparent effect is greater, we can be considerably less confident about thisestimate compared with that generated using a fixed-effect model. Table 7.19also shows an estimate of overall effect based on a random-effects modelafter accounting for publication bias. It is this final estimate that we wouldpresent in this report, not least because it is the most conservative in terms ofboth the effect size and confidence with which we can apply this effect to69