Deliverable D 1.3 RESULTS OF META-ANALYSIS ... - cast-eu.org

Campaigns and awareness raising strategies in traffic safety — Deliverable D-1.1normally distributed about the true effect size, which should be approachedwith greater proximity by those studies with greater weight. There is said to bepublication bias where this normal distribution is skewed in the direction of‘favourable’ effect.Figure 7.1 indicates that there is indeed publication bias for the set of effectsbased on accident counts included in the CAST database (bias can be seenin normal distribution of the solid diamonds). However, it is possible to correctfor this bias using a statistical method called trim-and-fill. This methodattempts to represent the true distribution of effects by adding in the ‘missing’or unfavourable data points. A computer algorithm is used to generate twoestimates of the number of studies that should be added to the ‘depleted’ sideof the distribution in order to make it normal. The two estimators are called Rand L.R is generated by first modifying the effect set by subtracting the average ofall effects from each individual effect. The mathematical sign of the modifiedeffects is then disregarded while they are ranked in order of size. Sign isreallocated to the resulting rank values, in order to find the number of effectsalong an uninterrupted sequence of one mathematical sign, beginning withthe largest effect value going down. The number 1 is subtracted from thenumber of effects in this uninterrupted sequence to obtain R.For the accident data in Figure 7.1 it is estimated that there are no missingstudies using R. This can be explained in simpler terms by referring to theextreme negative and positive ln effect values in Figure 7.1. Independent ofsign, the data point with the highest value is (-)1.31, but the next highest is(+)1.25, and so the number of ranks of one particular sign in the highestvalued sequence is 1. Thus according to the R estimator there are no missingstudies because 1-1=0.L estimates not the number of extreme values missing on the depleted side ofthe distribution, but the number of values missing across the depleted side ofthe effects. To begin with, the effects are again ranked according to size,independent of sign, and the signs of the original effects reattributed to theirrank values. The rank values are then divided into two sets, according to sign.Then the following equation is used 30 :L = (4T n – n[n+1]) / (2n-1)where T n is the sum of rank values in the largest rank set, and n is the totalnumber of effects in the group being analysed for bias . For the accident datain Figure 6.1, the estimator of missing effects L = 26. We can see by lookingat Figure 6.1 that, relative to the left, a number of effects are indeed missingacross the right hand side.Consultation of the two estimators R and L together is recommended by Duvaland Tweedie (2000). In our conservative approach, we simply use whicheverestimator gives the largest value.Trim-and-fill generates compensatory effects to fill the distribution by copyingthe most extreme values on the favoured side and ‘pasting’ them into the30 This is actually used in reiterative rounds of testing of the data set in order to achieve anoptimal solution, using a computer algorithm.67