Evaluating non-randomised intervention studies - NIHR Health ...

© Queen’s Printer and Controller of HMSO 2003. All rights reserved.Review Method of incorporating Results of quality investigationquality into synthesisNHS CRD, 1996 119 Qualitative Only good-quality CTs and case series were presented in the tables. Some methodological comment alsoprovided in the text. It was concluded that the quality of the research needs to be improvedNHS CRD, 1995 391 Not consideredNorman, 1998 205 Qualitative Three studies were rejected owing to low quality; the other seven studies were considered to have relativelyminor methodological problems. The interventions were found to be effective at undergraduate level, but not atresidency levelOakley, 1995 339 Qualitative Only studies judged ‘sound’ were used to evaluate effectiveness. Seven sound studies were reviewed of whichtwo showed short-term effects on reported sexual behaviour. No evidence to indicate that information provisionleads to risk-taking behaviour, but some indication that it may encourage sexual experimentationOakley, 1994 128 Qualitative Characteristics of ‘sound’ studies were discussed in more detail. The effectiveness of interventions was examinedfor both sound and flawed studies. Overall, only a small proportion of interventions were found to have beenevaluated in such a way that conclusions regarding effectiveness could be drawnOakley, 1995 234 Qualitative The conclusions from sound and flawed studies were compared; in general, a larger proportion of sound studiessuggested interventions to be beneficialOgilvie Harris, Qualitative Validity assessment results were presented and discussed but not really used to interpret study results. For1995 341 pharmacological studies around half of the studies were of reasonable quality, providing some evidence that theagents reviewed were better than placebo, but no evidence for one agent over any other. No qualityconsiderations in discussion of other interventions. Studies were examined in regard to the number of statisticallysignificant and non-significant conclusions presented, as there is no standardised method of scoring the severityof ankle injuries or the results of treatmentOxman, 1994 343 Not consideredOxman, 1994 271 Qualitative Quality described separately to results. The numbers of methodologically strong studies which provided data foreach outcome were reported. Overall five studies were found to be methodologically strong. Conclusions arelimited owing to paucity of well-designed studiesParker, 1998 398 Not consideredPloeg, 1996 232 Qualitative Quality features were not discussed in detail; however, authors conclude that the findings of the review shouldbe considered in the light of serious methodological limitations of the studiesPowe, 1994 399 Quantitative No association between visual acuity outcomes and quality score.Quality score and ESFor three complications, higher-quality studies reported higher proportions of complications than studies ofCompared high and low quality lower quality (p = 0.006, 0.02, 0.09)Weighted by quality scoreQuality score was used as a weight in the pooled analysis. Unweighted results were not presentedPuig Barbera, Quantitative Two studies excluded from meta-analysis for not meeting quality criteria; the impact of this on the results of the1995 345 Excluded lower quality studies meta-analysis was not discussedcontinuedHealth Technology Assessment 2003; Vol. 7: No. 27163