Statistical Methods in Medical Research 4ed

642 Clinical trials
original data. Pooling the data at the outset would unnecessarily reduce
precision if there were systematic differences in response between trials, and
bias might be caused if some trials departed from equal assignment proportions.
2 Although trials will never be exactly identical in treatments used, patient
characteristics, outcome variables, etc., they should be broadly similar in all
these respects. This is a vague recommendation, and differences of view as to
the choice of trials for inclusion are inevitable.
3 The trials should all involve random assignment and adequately masked
assessment. Protocol departures should be treated in the same way, preferably
by ITT.
4 All relevant and available data should be used.
Precept 4 is clearly open to different interpretations and practices. A widely
held view is that efforts should be made to retrieve unpublished data, partly to
avoid delays in publication, but more importantly to avoid publication bias
(§18.8), discussed further below. Completeness of coverage may be relatively
easy in some major areas of research, such as the acute treatment of myocardial
infarction, where all current research is known to senior practitioners, but will be
more difficult in less highly publicized areas. As noted in §18.8, a general system
for registration of trials, at their outset, would be beneficial (Simes, 1986b;
Dickersin et al., 1995).
Publication bias may sometimes be detected by a funnel plot, in which
an estimate of a parameter representing an efficacy difference for each trial is
plotted against the trial size (or some similar measure, such as the reciprocal of
the variance or standard error of the estimate). The estimates should cluster
round a constant value, with variability decreasing with trial size (and hence
producing the `funnel' shape). For small trial sizes, however, where non-significant
results are more likely, publication bias may cause the absence of points near
zero effect, and hence a skewness in the shape of the funnel. Examples, mainly
from epidemiology, are provided by Givens et al. (1997), who discuss methods of
adjusting for the bias. Other relevant work is by Dear and Begg (1992) and
Hedges (1992).
Many early meta-analyses worked merely with the summary statistics provided
in the published reports of each trial. Current practice favours the use of
individual patient data, enabling data checking and standardization (Stewart &
Parmar, 1993; Stewart & Clarke, 1995).
Different meta-analysis investigators are likely to differ in their
approach to some of the choices indicated in 1±4 above. It is, therefore,
not too surprising if different meta-analysis on ostensibly the same question
occasionally reach different conclusions (Chalmers et al., 1987). An example is
provided by two meta-analyses of trials comparing the anti-caries efficacy
of sodium fluoride and sodium monofluorophosphate dentifrices (Johnson,