Evaluating non-randomised intervention studies - NIHR Health ...

More documents

Recommendations

Info

Discussion and conclusionsNon-randomised studies provide a poor basis fortreatment or health policy decisions:●●The inability of case-mix adjustment methodsto compensate for selection bias and ourinability to identify non-randomised studieswhich are free of selection bias indicate thatnon-randomised studies should only be used insituations where RCTs cannot be undertaken.Healthcare policies based upon nonrandomisedstudies or systematic reviews ofnon-randomised studies may need re-evaluationif the uncertainty in the true evidence base wasnot fully appreciated when the decisions weremade.Recommendations for further research1. The resampling methodology that we haveemployed in this project should be applied inother clinical areas where suitable RCTs exist.These evaluations should consider (a) thedistribution of biases associated with nonrandomisedallocation, (b) whether nonrandomisedstudies with similar baselinecharacteristics are less biased and (c) theperformance of case-mix adjustment methods.It would be valuable to study different contextsto evaluate the degree to which bias is relatedto the amount of prognostic information knownat allocation.2. Efforts should be focused on the developmentof a new quality assessment tool for nonrandomisedstudies or the refinement anddevelopment of existing tools. Appropriatemethodological procedures of tooldevelopment should be employed and keyindicators of internal validity covered. Theseindicators include both those for whichempirical evidence is available from work onRCTs and those supported by our empiricalinvestigations. The latter should include themethod used to allocate participants to groups;specification of the factors that influenced theseallocations; the way in which these factors arethought to relate to outcome; and appropriateadjustment in the analysis. In the meantime,systematic reviewers should be stronglyencouraged to use and adapt those tools thatdo cover key quality issues.3. Research should be undertaken to developmethods of measuring and characterisingreasons for treatment choices in patientpreference and allocation by indication studies,and evaluations undertaken to assess whetherrecording such information allows effectiveadjustment for selection bias.4. Empirical work is needed to investigatehow quality assessments of non-randomisedstudies should be incorporated in thesynthesis of studies in a systematic reviewand to study the implications of individualquality features for the interpretation of reviewresults.5. Reasons for the failure of case-mix adjustmentmethods should be further investigated,including assessment of the generalisability ofour results to risk assessments andepidemiological studies where they arefrequently utilised. The impact of differencesbetween unconditional and conditionalestimates of treatment effects should beassessed.6. Guidelines should be produced to adviseinvestigators on the best ways of undertakingprospective non-randomised studies tominimise bias.7. The role of propensity scoring in adjusting forselection bias should be further evaluated, andcomputer macros made available for itsapplication.Recommendations for those producingand using health technologyassessments1. Systematic reviewers and those conductinghealth technology assessments should bestrongly encouraged to base any estimates ofeffectiveness or cost-effectiveness on evidencefrom RCTs. Where such evidence is unavailable,the uncertainty inherent in estimates based onnon-randomised evidence should be stronglyemphasised.2. Decision-makers should review healthcarepolicies based on the results of non-randomisedstudies to assess whether the inherentuncertainty in the evidence base was properlyappreciated when the policy decisions weremade.3. Agencies funding primary research should fundnon-randomised studies only when they areconvinced that a randomised study is notfeasible.92
Health Technology Assessment 2003; Vol. 7: No. 27AcknowledgementsWe would like to thank the following peoplefor their help with this project: PeterSandercock and Peter Rothwell for organisingaccess to the IST and ECST data setsrespectively; Jane Harrison for carrying outthe extensive set of searches and Kath Wrightfor help with search queries; Alison Brown andVanda Castle for secretarial support. We wouldalso like to acknowledge the helpful commentsof the members of our expert panel:Professor Brian Haynes, Professor Nick Black andDr Barnaby Reeves.93© Queen’s Printer and Controller of HMSO 2003. All rights reserved.
Page 1 and 2:
Health Technology Assessment 2003;
Page 3 and 4:
Evaluating non-randomisedinterventi
Page 5 and 6:
Page 7:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17:
Page 21 and 22:
Page 23 and 24:
© Queen’s Printer and Controller
Page 25 and 26:
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33:
Page 36 and 37:
Evaluation of checklists and scales
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
Page 44 and 45:
32TABLE 8 Details of top 60 quality
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
38TABLE 10 Other domains: reporting
Page 52 and 53:
Page 54 and 55: Evaluation of checklists and scales
Page 56 and 57: Use of quality assessment in system
Page 62 and 63: Empirical estimates of bias associa
Page 76 and 77: Empirical evaluation of the ability
Page 82 and 83: 70TABLE 22 Comparison of concurrent
Page 86 and 87: 74TABLE 26 Comparison of methods of
Page 92 and 93: 80TABLE 33 Hypothetical example dem
Page 94 and 95: 82TABLE 34 Hypothetical example dem
Page 100 and 101: Discussion and conclusions88histori
Page 102 and 103: Discussion and conclusions90For exa
Page 107 and 108: Health Technology Assessment 2003;
Page 121: Health Technology Assessment 2003;
Page 124 and 125: Appendix 1data)) or (non-random$ or
Page 126 and 127: Appendix 2AuthorYearENDARESourcePub
Page 128 and 129: Appendix 2Author:Accession No:Endno
Page 130 and 131: Appendix 20 0 00Additional outcomes
Page 132 and 133: Appendix 2Endnote NoWas CMA conside
Page 134 and 135: 122AuthorOrigin aModified toolTool
Page 148 and 149: Appendix 4136DuRant, 1994 99The typ
Page 155 and 156:
Page 157 and 158:
Page 159 and 160:
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185:
Page 188 and 189:
Health Technology Assessment Progra
Page 190:
Health Technology Assessment Progra
show all

Evaluating non-randomised intervention studies - NIHR Health ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?