Evaluating non-randomised intervention studies - NIHR Health ...

More documents

Recommendations

Info

152Review Method of incorporating Results of quality investigationquality into synthesisACCP/AACVPR, Qualitative Studies were classified according to level of evidence and recommendations were graded accordingly.1997 393 Methodological issues were discussed narrativelyAnders, 1996 81 Quantitative The t-test of quality score differences was non-significant (p = 0.25). The authors conclude that they cannotSubgroup analysis according to size of effect; reject the null hypothesis; no difference in the quality score between studies existedbetween group differences in mean qualityscores investigatedAronson, 1996 249 Quantitative Pooling all studies indicated a strongly positive effect (RR 2.09, 95% CI: 1.31 to 3.32, p = 0.002). Eliminating oneSensitivity analysis sequentially removing NRS with no control group and then one RCT with a treated control group lowered the relative response togroups of studies with sharedtreatment (RR 1.93, 95% CI: 1.13 to 3.29, p = 0.02). Further elimination of an RCT with possibly contaminatedmethodological deficienciesrandomisation increased the relative response to treatment (RR 2.70, 95% CI: 1.89 to 3.86, p = 0.006), andeliminated the statistical heterogeneity of the results. The three NRS which remained in the analysis were HCTsand therefore likely to have biased resultsAudet, 1993 176 Qualitative Study results were discussed in terms of methodological quality. Only two studies were of sufficiently high qualityto demonstrate a positive effect from teachingBaker, 1994 82 Quantitative Two higher quality studies indicated statistically significant reductions in mortality; of the five others, three hadForest plot used to present study results statistically significant reductions and two were non-significantranked by qualityBass, 1993 88 Qualitative Studies grouped by design, and quality score and overall rank presented: some NRS ranked more highly thanRCTs. Found reasonably high-quality evidence to support the interventionBauman, 1997 208 Qualitative A statement was made regarding the overall quality of included studies. Quality stated to be poor, limiting theinterpretation of resultsBerard, 1997 209 Quantitative The correlation between effect size and quality score (r = –0.07, p = 0.19) or sample size (r = 0.04, p = 0.63)Correlation analysis to investigate ES and were not found to be significantquality scoreBeyer, 1998 210 Quantitative When all studies were pooled, a small, positive, but non-significant effect from influenza vaccine was producedSubgroup analysis according to(RD –0.6, 95% CI: –3 to 1.9). The two poor-quality studies (both of the NRS) showed a much more stronglyquality (high/medium/low)protective effect from the vaccine (RD –9.4, no CIs given), and the medium- and high-quality studies (all RCTs)indicated a much more conservative effect (RD –0.3 and 1.1, respectively, no CIs given)Boers, 1991 212 Qualitative Methodological characteristics were discussed in detail and studies were categorised and discussed according tostrength of evidence. Insufficient evidence was found to make any recommendations regarding drugcombinationsBours, 1998 73 Quantitative Three studies with only positive results had lower quality scores; those with only negative results had similarVisual plot used to map relationship between scores to those with mixed (+ve/–ve) outcomes. Studies with highest scores demonstrated no effect. Five NRSstudy outcome and methodological score scored more highly than any RCTAppendix 7continued
© Queen’s Printer and Controller of HMSO 2003. All rights reserved.Review Method of incorporating Results of quality investigationquality into synthesisBrown, 1988 254 Quantitative No significant correlations with unweighted mean effect sizes found. Experimental mortality (attrition) was the38 substantive and methodological only variable significantly correlated with the overall weighted mean ES, r = –0.52, p = 0.002characteristics were investigated in acorrelational analysisBrown, 1990 215 Quantitative Correlation analysis found no relation between attrition rate or rating of research quality and mean ES estimates.Correlational analysis of ES and quality score. Lower-quality studies produced smaller ESs for three of the outcome measures and larger ESs for the other two.Subgroup analyses according to quality For four out of five outcomes the SDs associated with non-equivalent control groups were almost twice as(high/low) and rigour of design for five large as for the other designsoutcome measuresBrown, 1995 216 Not consideredBrown, 1996 217 Quantitative None of the relationships were statistically significant. Strongest relationship was with ES for behavioural strategyCorrelation of quality index and effect sizes (r = –0.46, p = 0.07, n = 16); the higher the quality the lower the ES for mean weight loss. Outcome effectsalso partitioned according to study design: all results were statistically significant for pre-/post-test studies(approx. twofold higher), but for experimental studies, only metabolic control variables were statisticallysignificant. No data presentedBurckhardt, 1987 362 Quantitative High internal validity, high/adequate reliability of the outcome measure, and high reactivity of outcome measuresSubgroup analysis according to high/medium demonstrated larger ES (0.60, 0.59, and 0.80 respectively compared with overall ES of 0.53). No informationvalidity; reliability and reactivity of the was provided regarding the statistical significance of these subgroup analyses. Authors comment that theoutcome measuresignificance of study design characteristics is unclearCallahan, 1995 264 Quantitative No significant correlations were with publication year, mean follow-up, sample size, or % lost to FU.Correlation analysis used to determineif study characteristics were correlatedwith outcomesCallahan, 1991 218 Quantitative Inclusion of lower quality studies marginally increased the overall effect and slightly narrowed the CIs. ResultsOnly studies meeting all quality criteria were not presented for low-quality studies only. Treatment ES was also examined visually in relation to sampleincluded in main analysis; qualifying criteria size for studies meeting all quality criteria and the five studies meeting some criteria. The inclusion of lowerwere sequentially relaxedquality studies did not significantly alter the appearance of the graphCameron, 2000 83 Qualitative Little methodological discussion on included studies; however authors conclude that “the available data havesignificant limitations in quantity and quality and allow only tentative conclusions”, namely that there is limitedevidence for the intervention in terms of hospital admission and residential status following discharge. Mainanalyses focus on comparing RCT and NRS resultsCamma, 1996 335 Qualitative Methodological limitations of studies were discussed. Quality score was not used in the meta-analysis. Authorscomment that quality was reasonably good.Cao, 1997 226 Quantitative Weighting effect size by both quality score and sample size made essentially no difference to the overall results.Quality weightingOnly sample size weighted results presentedcontinuedHealth Technology Assessment 2003; Vol. 7: No. 27153
Page 1 and 2:
Health Technology Assessment 2003;
Page 3 and 4:
Evaluating non-randomisedinterventi
Page 5 and 6:
Page 7:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17:
Page 21 and 22:
Page 23 and 24:
© Queen’s Printer and Controller
Page 25 and 26:
Page 27 and 28:
© Queen’s Printer and Controller
Page 29 and 30:
Page 31 and 32:
Page 33:
Page 36 and 37:
Evaluation of checklists and scales
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
Page 44 and 45:
32TABLE 8 Details of top 60 quality
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
38TABLE 10 Other domains: reporting
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
Use of quality assessment in system
Page 58 and 59:
Page 60 and 61:
Page 62 and 63:
Empirical estimates of bias associa
Page 64 and 65:
Page 66 and 67:
Page 68 and 69:
Page 70 and 71:
Page 72 and 73:
Page 74 and 75:
Page 76 and 77:
Empirical evaluation of the ability
Page 78 and 79:
Page 80 and 81:
Page 82 and 83:
70TABLE 22 Comparison of concurrent
Page 84 and 85:
Page 86 and 87:
74TABLE 26 Comparison of methods of
Page 88 and 89:
Page 90 and 91:
Page 92 and 93:
80TABLE 33 Hypothetical example dem
Page 94 and 95:
82TABLE 34 Hypothetical example dem
Page 96 and 97:
Page 98 and 99:
Page 100 and 101:
Discussion and conclusions88histori
Page 102 and 103:
Discussion and conclusions90For exa
Page 104 and 105:
Discussion and conclusionsNon-rando
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114: Health Technology Assessment 2003;
Page 121: Health Technology Assessment 2003;
Page 124 and 125: Appendix 1data)) or (non-random$ or
Page 126 and 127: Appendix 2AuthorYearENDARESourcePub
Page 128 and 129: Appendix 2Author:Accession No:Endno
Page 130 and 131: Appendix 20 0 00Additional outcomes
Page 132 and 133: Appendix 2Endnote NoWas CMA conside
Page 134 and 135: 122AuthorOrigin aModified toolTool
Page 148 and 149: Appendix 4136DuRant, 1994 99The typ
Page 155 and 156: © Queen’s Printer and Controller
Page 188 and 189: Health Technology Assessment Progra
Page 190: Health Technology Assessment Progra
show all

Evaluating non-randomised intervention studies - NIHR Health ...

Create successful ePaper yourself

Delete template?

Save as template?