Evaluating non-randomised intervention studies - NIHR Health ...

More documents

Recommendations

Info

Use of quality assessment in systematic reviews of non-randomised studiesSystematic reviewsscreened1162Excluded:Diagnostic studies: 94RCTs only: 510Uncontrolled only: 47Reviews with NRS511No QA repoted343Quality assessed168Used existingQA tool69Modified existingQA tool28Developed ownQA tool71Tools usedNo. of reviewsChalmers: 6Maastricht Criteria List: 3Jadad scale: 3Gyorkos tool: 3Oakley: 3Haynes: 3Other (all
Health Technology Assessment 2003; Vol. 7: No. 27TABLE 11 Breakdown of reviews including non-randomised studiesStudy designs included No. of reviews No. using quality Proportion using qualityassessment assessment (%)NRS only 32 5 16NRS + RCTs 216 99 46NRS + RCTs + uncontrolled 116 34 29NRS + uncontrolled 41 9 22Not clearly reported 106 21 20Total 511 168 33NRS, non-randomised study.least one RCT and in 87 reviews a term such as‘controlled trial’ was used to describe the studiesso that RCTs were not clearly distinguished fromnon-randomised studies (Table 11). Those reviewswhich included both RCTs and non-randomisedstudies were more likely to have conducted qualityassessment (conducted in 46% of cases) than thosewith non-randomised studies only (16%) or thosealso including uncontrolled studies. Full details ofthe review methods are provided in Appendix 6and review results in Appendix 7.Source of assessment toolsOf the 168 reviews claiming to have carried outquality assessment, 69 (41%) stated they had usedexisting quality assessment tools, 28 (17%) mademodifications to existing tools and 71 (42%)developed their own tool (Figure 6).Closer inspection of the papers cited in the 69reviews that used existing tools revealed that 10 ofthe tools (cited in 13 of the reviews) were not infact quality assessment tools, but weremethodological papers, textbooks or manuals onhow to conduct reviews 118–123 or design primarystudies. 50,124,125 Nineteen reviews used a variety ofversions of the ‘levels of evidence’ frameworkoriginally developed by Sackett 126 and did notassess quality beyond identifying the study design.Amongst the remaining 37 reviews, the mostfrequently used tools were originally developed forthe assessment of RCTs (Figure 6): the Chalmersscale, 51 or modifications of it 76,77,92,96 (six reviews);the Maastricht Criteria List 69,70 (three reviews);and the Jadad scale 46 (three reviews). Othercommonly used tools were ones developed for usein public health 127 (used in three reviews) and atool developed by Oakley and colleagues 128developed for and used by the same authors forreviews of the health promotion literature (threereviews).Of the 28 reviews that modified existing tools,only one 26 reported details about how the tool(originally published by Downs and Black 85 ) hadbeen modified and also attempted to examine thevalidity of the tool. Twelve of these reviewsreported inter-rater reliability, all producing kappacoefficients of >0.70 or percentage agreement ofat least 80%.In the review by MacLehose and colleagues, 26 theitems considered to be the most important andunambiguous ‘quality items’ were identified bythree authors and were categorised into one offour dimensions: quality of reporting, internalvalidity, susceptibility to confounding and externalvalidity. The internal consistency of the individualitems contributing to the first three of thesedimensions was described using Cronbach’s α.The resulting low α values indicated thatindividual items were poorly correlated with thesum of scores for the items, suggesting that theitems were not assessing a homogeneous aspect ofquality. An attempt was also made to validate thefour quality dimensions and the sum of the fourdimensions by investigating predictors of qualityscores. Study design, intervention and theinteraction of study design and intervention wereentered into the regression model. For totalquality scores, both cohort and case–controlstudies had significantly poorer quality than RCTs.There was no independent effect of interventionand no interaction of intervention and studydesign. Case–control studies scored more highlythan cohort studies.For the 71 tools developed by review authors, nodetails of the tool development were provided andnone justified the choice of quality items. Twentyonereviews examined inter-rater reliability andfound similar levels of agreement to thosediscussed previously. In addition, two furtherreviews 93,129 addressed the validity of their tools.45© Queen’s Printer and Controller of HMSO 2003. All rights reserved.
Page 1 and 2:
Health Technology Assessment 2003;
Page 3 and 4:
Evaluating non-randomisedinterventi
Page 5 and 6: Health Technology Assessment 2003;
Page 7: Health Technology Assessment 2003;
Page 23 and 24: © Queen’s Printer and Controller
Page 27 and 28: © Queen’s Printer and Controller
Page 36 and 37: Evaluation of checklists and scales
Page 44 and 45: 32TABLE 8 Details of top 60 quality
Page 50 and 51: 38TABLE 10 Other domains: reporting
Page 58 and 59: Use of quality assessment in system
Page 60 and 61: Use of quality assessment in system
Page 62 and 63: Empirical estimates of bias associa
Page 76 and 77: Empirical evaluation of the ability
Page 82 and 83: 70TABLE 22 Comparison of concurrent
Page 86 and 87: 74TABLE 26 Comparison of methods of
Page 92 and 93: 80TABLE 33 Hypothetical example dem
Page 94 and 95: 82TABLE 34 Hypothetical example dem
Page 100 and 101: Discussion and conclusions88histori
Page 102 and 103: Discussion and conclusions90For exa
Page 104 and 105: Discussion and conclusionsNon-rando
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
Page 117 and 118:
Page 119 and 120:
Page 121:
Page 124 and 125:
Appendix 1data)) or (non-random$ or
Page 126 and 127:
Appendix 2AuthorYearENDARESourcePub
Page 128 and 129:
Appendix 2Author:Accession No:Endno
Page 130 and 131:
Appendix 20 0 00Additional outcomes
Page 132 and 133:
Appendix 2Endnote NoWas CMA conside
Page 134 and 135:
122AuthorOrigin aModified toolTool
Page 136 and 137:
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Appendix 4136DuRant, 1994 99The typ
Page 151 and 152:
Page 153 and 154:
Page 155 and 156:
© Queen’s Printer and Controller
Page 157 and 158:
Page 159 and 160:
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185:
Page 188 and 189:
Health Technology Assessment Progra
Page 190:
Health Technology Assessment Progra
show all

Evaluating non-randomised intervention studies - NIHR Health ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?