Evaluating non-randomised intervention studies - NIHR Health ...

More documents

Recommendations

Info

162Review Method of incorporating Results of quality investigationquality into synthesisMayo-Smith, 1997 241 Not consideredMaziak, 1998 192 Qualitative Quality issues were discussed alongside the results of each study. A lack of rigorously controlled trials wasidentified and the reviewed evidence provides insufficient information to permit a reasonable conclusion to bemadeMcAweeney, 1997 325 Not consideredMcCusker, 1998 328 Quantitative Results stated to be similar for quality-adjusted and unadjusted effect measures. Only quality-adjusted resultsQuality weightingpresentedMelchart, 1994 93 Qualitative Methodological issues and results discussed. Reviewer’s estimate of strength of evidence provided by each studywas presented. Inconclusive or limited evidence of no efficacy was provided by 21 comparisons and limited orgood evidence of efficacy was provided by 10 comparisonsMiller, 1995 87 Quantitative No significant relation between these codes and methodological quality score was found (r = 0.06). QualityCorrelation analysis of quality score and score was also found to be unrelated to mean problem severity and to estimated cost of delivering the‘outcome logic scores’ (measure of the treatmentstrength of support for treatment efficacy)Morin, 1994 330 Quantitative Design quality was negatively correlated with ES for sleep onset latency (r = –0.24, df = 91, p < 0.03) only,Correlations between outcome and quality indicating that lower quality studies produced larger ESsof study design were calculatedMullen, 1985 275 Quantitative No impact on ES foundRegression analysis to examine effect ofstudy design and type of comparison groupMullen, 1992 274 Qualitative Some narrative discussion of methodological characteristics. Pre-/post-test studies were excluded from the metaanalysisowing to methodological limitations. Further implications of methodological flaws not discussed in any detailMulrow, 1986 197 Qualitative Some quality issues were discussed alongside study results. Authors conclude that methodological limitations donot allow many definitive conclusions but that some new and useful information was obtainedMurtagh, 1995 332 Quantitative Most effect were non-significant – only the source of participants, non-use of drugs and placebo treatmentDesign variables used as inputs into a contributed significantly to the prediction of the dependent variableregression to quantify impact on effect sizes(including source of sample, study design, useof objective validation)Naylor, 1987 334 Quantitative When all studies were combined (3 RCTs and 1 NRS), quality weighting abolished the significant treatment effectSensitivity analysis conducted using weighting for both overall survival from acute episode and overall survival to discharge from hospital. (Inclusion of the NRSby quality score. Weighted and unweighted led to larger treatment effects). Authors conclude that despite the (unweighted) significant findings, there isp-value for the difference between groups insufficient evidence for the interventionwas providedAppendix 7continued
© Queen’s Printer and Controller of HMSO 2003. All rights reserved.Review Method of incorporating Results of quality investigationquality into synthesisNHS CRD, 1996 119 Qualitative Only good-quality CTs and case series were presented in the tables. Some methodological comment alsoprovided in the text. It was concluded that the quality of the research needs to be improvedNHS CRD, 1995 391 Not consideredNorman, 1998 205 Qualitative Three studies were rejected owing to low quality; the other seven studies were considered to have relativelyminor methodological problems. The interventions were found to be effective at undergraduate level, but not atresidency levelOakley, 1995 339 Qualitative Only studies judged ‘sound’ were used to evaluate effectiveness. Seven sound studies were reviewed of whichtwo showed short-term effects on reported sexual behaviour. No evidence to indicate that information provisionleads to risk-taking behaviour, but some indication that it may encourage sexual experimentationOakley, 1994 128 Qualitative Characteristics of ‘sound’ studies were discussed in more detail. The effectiveness of interventions was examinedfor both sound and flawed studies. Overall, only a small proportion of interventions were found to have beenevaluated in such a way that conclusions regarding effectiveness could be drawnOakley, 1995 234 Qualitative The conclusions from sound and flawed studies were compared; in general, a larger proportion of sound studiessuggested interventions to be beneficialOgilvie Harris, Qualitative Validity assessment results were presented and discussed but not really used to interpret study results. For1995 341 pharmacological studies around half of the studies were of reasonable quality, providing some evidence that theagents reviewed were better than placebo, but no evidence for one agent over any other. No qualityconsiderations in discussion of other interventions. Studies were examined in regard to the number of statisticallysignificant and non-significant conclusions presented, as there is no standardised method of scoring the severityof ankle injuries or the results of treatmentOxman, 1994 343 Not consideredOxman, 1994 271 Qualitative Quality described separately to results. The numbers of methodologically strong studies which provided data foreach outcome were reported. Overall five studies were found to be methodologically strong. Conclusions arelimited owing to paucity of well-designed studiesParker, 1998 398 Not consideredPloeg, 1996 232 Qualitative Quality features were not discussed in detail; however, authors conclude that the findings of the review shouldbe considered in the light of serious methodological limitations of the studiesPowe, 1994 399 Quantitative No association between visual acuity outcomes and quality score.Quality score and ESFor three complications, higher-quality studies reported higher proportions of complications than studies ofCompared high and low quality lower quality (p = 0.006, 0.02, 0.09)Weighted by quality scoreQuality score was used as a weight in the pooled analysis. Unweighted results were not presentedPuig Barbera, Quantitative Two studies excluded from meta-analysis for not meeting quality criteria; the impact of this on the results of the1995 345 Excluded lower quality studies meta-analysis was not discussedcontinuedHealth Technology Assessment 2003; Vol. 7: No. 27163
Page 1 and 2:
Health Technology Assessment 2003;
Page 3 and 4:
Evaluating non-randomisedinterventi
Page 5 and 6:
Page 7:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17:
Page 21 and 22:
Page 23 and 24:
© Queen’s Printer and Controller
Page 25 and 26:
Page 27 and 28:
© Queen’s Printer and Controller
Page 29 and 30:
Page 31 and 32:
Page 33:
Page 36 and 37:
Evaluation of checklists and scales
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
Page 44 and 45:
32TABLE 8 Details of top 60 quality
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
38TABLE 10 Other domains: reporting
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
Use of quality assessment in system
Page 58 and 59:
Page 60 and 61:
Page 62 and 63:
Empirical estimates of bias associa
Page 64 and 65:
Page 66 and 67:
Page 68 and 69:
Page 70 and 71:
Page 72 and 73:
Page 74 and 75:
Page 76 and 77:
Empirical evaluation of the ability
Page 78 and 79:
Page 80 and 81:
Page 82 and 83:
70TABLE 22 Comparison of concurrent
Page 84 and 85:
Page 86 and 87:
74TABLE 26 Comparison of methods of
Page 88 and 89:
Page 90 and 91:
Page 92 and 93:
80TABLE 33 Hypothetical example dem
Page 94 and 95:
82TABLE 34 Hypothetical example dem
Page 96 and 97:
Page 98 and 99:
Page 100 and 101:
Discussion and conclusions88histori
Page 102 and 103:
Discussion and conclusions90For exa
Page 104 and 105:
Discussion and conclusionsNon-rando
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
Page 117 and 118:
Page 119 and 120:
Page 121:
Page 124 and 125: Appendix 1data)) or (non-random$ or
Page 126 and 127: Appendix 2AuthorYearENDARESourcePub
Page 128 and 129: Appendix 2Author:Accession No:Endno
Page 130 and 131: Appendix 20 0 00Additional outcomes
Page 132 and 133: Appendix 2Endnote NoWas CMA conside
Page 134 and 135: 122AuthorOrigin aModified toolTool
Page 148 and 149: Appendix 4136DuRant, 1994 99The typ
Page 151 and 152: Health Technology Assessment 2003;
Page 155 and 156: © Queen’s Printer and Controller
Page 173: © Queen’s Printer and Controller
Page 185: Health Technology Assessment 2003;
Page 188 and 189: Health Technology Assessment Progra
Page 190: Health Technology Assessment Progra
show all

Evaluating non-randomised intervention studies - NIHR Health ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?