Evaluating non-randomised intervention studies - NIHR Health ...

© Queen’s Printer and Controller of HMSO 2003. All rights reserved.Review Method of incorporating Results of quality investigationquality into synthesisEPI Centre, 1996 338 Qualitative All studies were discussed in terms of methodology, but only the results of the 21 methodologically sound studieswere discussed in detail. There was disagreement between authors and reviewers regarding effectiveness ofinterventions in 24% of methodologically ‘sound’ studiesErnst, 1998 287 Qualitative Authors focused only on highest quality studies (both scored 3). One found some increase in pain threshold andanother found no effect. Authors’ concluded that there is evidence of benefitErnst, 1998 302 Qualitative Three higher quality studies included (all RCTs); all reported no statistically significant differences; lower qualitytrials indicated some benefit from homeopathyFiscella, 1995 267 Qualitative Evidence was examined in regard to methodological criteria. Authors conclude that current evidence does notsatisfy causal criteria necessary to establish that prenatal birth care definitely improves birth outcomesFlint, 1995 326 Qualitative Some methodological details were provided in the discussion of each individual study. Author concludes thatthere is little evidence for the intervention but that this may be due to methodological limitationsFloyd, 1997 371 Quantitative Not reported in detail; only one variable (use of pretraining) stated to be significantly correlated with effect sizeCorrelation analysis to examine relationshipbetween study and sample characteristics andoutcomeFoxcroft, 1997 340 Qualitative Quality issues discussed along with study results. Most had some methodological shortcomings. No oneintervention could be recommendedGam, 1995 229 Quantitative Impact depended on type of ES used: when d/r was used there was no influence from blinding (p = 0.78), whileSubgroup analysis to examine effect of the d/s showed significant influence (p = 0.009) (d/r and d/s stated to be standardised effect sizes)blindingGansevoort, 1995 261 QuantitativeRegression analysis to examine patient andstudy characteristics and effect on primaryoutcomeGarg, 1998 263 Not consideredStudy design (randomised, double blind or cross-over) made no essential difference to the results. The mostimportant variables were related to the intervention used or clinical characteristics of the populationGlantz, 1997 182 Quantitative Restricting the analysis to only high-quality studies (two RCTs and one NRS) made little difference to theSubgroup analysis of high quality studies only summary effect: the OR increased from 0.61 (95% CI: 0.50 to 0.75) to 0.66 (95% CI: 0.54 to 0.81), indicating alower reduction in the odds of caesarean due to active management of labour. The authors also presentedquality-adjusted results for RCTs only (adjusted by a factor of 2), which further reduced the odds of caesarean:OR 0.70 (95% CI: 0.57 to 0.78). The authors recognise that the larger effect demonstrated when all studieswere combined may be due to the fact all three NRS were HCTs.Good, 1996 266 Qualitative The validity of the results was questioned owing to numerous methodological problems in the studiescontinuedHealth Technology Assessment 2003; Vol. 7: No. 27157