Evaluating non-randomised intervention studies - NIHR Health ...

154Review Method of incorporating Results of quality investigationquality into synthesisCarlson, 1996 227 Quantitative All correlations were non-significant. Less than 1% of the ES variability was associated with the quality of studyCorrelation analysis between ES and six design variable (r 2 = 0.003)potential moderator variables (year ofpublication, study design, residency ofsubjects, mean age, sample size and durationof treatment)Carter, 1994 178 Qualitative Results used to indicate that existing research is methodologically weakCatre, 1997 222 Not consideredChaulk, 1998 240 Not consideredCheatham, 1995 230 Quantitative In most cases, high-quality studies produced higher relative risk estimates, although the magnitude of theSubgroup analysis of high quality studies difference in effect varied. In three cases restriction to high quality studies changed the statistical significance of(15 RCTs and five case–control studies) alone the results; in two cases a non-significant result became statistically significant, and in the other the pooled resultfor 15 outcome variablesbecame non-significantCheek, 1998 130 Qualitative Methodological limitations severely limited the conclusions that could be drawn from the literature, however,some recommendations were madeChoi, 1995 352 Quantitative Studies using a placebo control group generally led to higher pooled effects than those with a no treatmentSubgroup analysis according to use of control group (incidence rate ratios 0.801 vs 0.626, respectively)placebo-control groupWeighting by quality score produced a lower rate ratio compared with the unadjusted results; e.g. for placebo-Presentation of results with and without controlled studies, the rate ratios fell from 0.801 (95% CI: 0.693 to 0.926) to 0.757 (95% CI: 0.611 to 0.937).weighting by quality scoreFor the analysis of permethrin-treated bed nets only (n = 9) the effect of using a placebo control group and ofweighting by quality score were not as strikingCiliska, 1996 231 Qualitative Quality (strong/weak, etc.) was discussed alongside the results of each study. Overall, despite limitations in theresearch, the authors conclude that sufficient high-quality studies are available to demonstrate a positive directionof effectCiliska, 1995 351 Qualitative Weak studies excluded from the review. Some methodological characteristics of included studies were discussed.Authors conclude that the evidence is equivocalCohen, 1994 236 Qualitative Only higher scoring studies were used to ascertain whether findings were related to characteristics of studyparticipants, interventions or settings (n = 6). Little sound evdience was found to support the effect of groupeducation on short-term outcomes (3-month FU), and no evidence of an effect at long-term FU (1 year)Colditz, 1994 237 Quantitative Quality explained 30% of the variation in BCG efficacy in the clinical trials; geographic altitude individuallyRegression analysis to examine quality score explained 41% of the variance. The authors could not determine whether these variables were surrogates forand ESfactors such as quality of FU or presence of non-tuberculous mycobacteria in the population. Quality explained36% of the between-study heterogeneity in the case–control studies, and was the only variable to have anyimpactAppendix 7continued