Evaluating non-randomised intervention studies - NIHR Health ...

More documents

Recommendations

Info

134AuthorOrigin aModified toolTool purpose bStudy designscovered cType of tool d(weighting)Items justified?Items by design?No. of itemsTool validity eInter-raterreliability fIntra-raterreliability fTest-retest fInternalconsistency fTime to completeUsed in otherreviews?Review referencenumberAppendix 3Vickers, 1995 110 n 3 CCTs c n n 21 na na na na na n nVickers, 1994 386 n 7 RCTs, Coh c n n NR na na na na na n nVickers, 1996 201 n See also Vickers, 7 RCTs, NRS c n n 12 na “good” na na na n n1994 386Ward, 1979 387 n 3 Any c n n 53 na na na na na n nWeiler, 1992 388 n 7 RCTs, CTs c n n 14 na na na na na n yWeintraub, 1982 108 m Lionel, 1970 315 1/2/3 CCTs c n n 47 na na na na na n nWells-Parker, 1995 129 n 7 RCTs, NRS s e n n ? FC grouping strategies na na na n ndomain: 0.95 rWilson, 1992 103 n 7 RCTs, Coh, CCS c n y 10 na na na na na n nWingood, 1996 202 n 7 RCTs, NRS c n n 16 na na na na na n nWolf, 1993 389 n 5 Any s e n n 31 na na na na na n nWoolf, 1990 390 L Can Task Force, 4 y 3911979 219Wright, 1995 203 n 7 RCTs, NRS c n n 13? na na na na na n nZaza, 2000 86 n 7 RCTs, NRS c n n 22 na na na na na n nZola, 1989 392 m Chalmers, 1981 51 2 RCTs, NRS; Coh c n n 11 na na na na na n na Origin: n, new; m, modified; X, not a quality assessment tool; L, levels of evidence classification system.b Tool purpose: 1, planning a study; 2, assessing a statistical/methodological analysis; 3, evaluating a study when considering the practical application of the intervention; 4, evaluatingor grading study recommendations when producing guidelines; 5, peer reviewing; 6, reporting a study; 7, assessing studies included in a systematic review (see Box 3).c Study designs covered: RCT, randomised controlled trial; CCT, controlled clinical trial; Coh, cohort; CCS, case–control; Exp, experimental; B/A, before and after; Pre/Post,pre/post; Obs, observational; CS, case series (or cross-sectional); FU, follow-up studies; NRS, non-randomised studies.d Type of tool: c, checklist; s, scale; u, unequal weighting; e, equal weighting.e Tool validity assessments: FC, face and content; Cr, criterion; Co, content; Fa, factorial.f Tool reliability assessments: IRR, inter-rater reliability; IaRR, intra-rater reliability; IC, internal consistency; T-R, test-retest; K, kappa; W, Kendall’s W; a , % agreement; r , intra-classcorrelation; KR, Kuder–Richardson formula 20; CA, Cronbach’s alpha; P, Pearson correlation; α , coefficient alpha; C, clinician; M, methodologist; SE, standard error. NB: low internalconsistency suggests that the criteria are relatively easy to satisfy.na, not assessed; NR, not reported.
Health Technology Assessment 2003; Vol. 7: No. 27Appendix 4Detailed description of ‘unselected’top 14 quality assessment toolsBracken, 1989 104The types of questions which led to the selectionof this tool according to our pre-selected corecriteria were:5.3 How allocation occurredIf historical rather than concurrent controlshave been used, has it been explained whythis does not introduce bias into the study?5.4 Any attempt to balance groups by designIf study groups were ‘matched’, have therationale and detailed criteria for matching,and success (i.e. number of cases not matched)been provided?9.2 Identification of prognostic factorsHas the measurement of importantconfounding or effect-modifying variablesbeen described so the reader can judge howthey have been controlled?9.3 Case-mix adjustmentHas an analysis of potential confounding oreffect modification of the principal relationsbeen presented?This tool provides a list of 36 questions to guidethe reporting of observational studies. The tool issplit into four sections: introduction to the report(4 items); description of materials and methods(17 items); presentation of results (7 items); andstudy conclusions (8 items). The tool took 20–30minutes to complete. It was clearly not designedfor use in a systematic review and to ourknowledge has not been used for that purpose.The majority of the questions could be answeredyes/no, making it possible to compare responsesacross studies. However, the questions are notphrased in such a way that any real judgements ofthe methodological quality of a study could bemade, as the questions listed above demonstrate.Furthermore, the study designs that the tool couldcover were limited to case-control and cohortdesigns.Critical Appraisal SkillsProgramme, 1999 64The types of questions which led to the selectionof this tool according to our pre-selected corecriteria were:5.3 How allocation occurredItem not covered5.4 Any attempt to balance groups by designHave the authors taken account of theconfounding factors in the design and/oranalysis?9.2 Identification of prognostic factorsWhat factors could potentially be related toboth the exposure and the outcome of interest(confounding)?9.3 Case-mix adjustmentSame item as 5.4 aboveThis tool provides a list of 12 questions to aid thecritical appraisal of cohort studies. The tool is splitinto three sections: ‘are the results of the studyvalid?’ (7 items); ‘what are the results?’ (2 items)and ‘will the results help locally?’ (2 items). Thetool took 15–20 minutes to complete. It was notused in our sample of systematic reviews, and in itscurrent format would be difficult to apply in sucha context. The questions prompt thinking aboutquality but were felt to require subjective answersthat are unlikely to be consistent within or betweenreviewers. The questions are followed by ‘hints’ tohelp the reader answer the overall questions, butin some cases it is difficult to work out how theresponses to the hints translate into yes/no/can’ttell. For example, if a reviewer answered ‘yes’ totwo hints and ‘no’ to two hints for a singlequestion, it is not clear what the overall answer tothat question should be.The CASP tool was not judged to be suitable foruse in a systematic review in its current format.The Bracken tool was not judged to be suitable foruse in a systematic review.135© Queen’s Printer and Controller of HMSO 2003. All rights reserved.
Page 1 and 2:
Health Technology Assessment 2003;
Page 3 and 4:
Evaluating non-randomisedinterventi
Page 5 and 6:
Page 7:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17:
Page 21 and 22:
Page 23 and 24:
© Queen’s Printer and Controller
Page 25 and 26:
Page 27 and 28:
© Queen’s Printer and Controller
Page 29 and 30:
Page 31 and 32:
Page 33:
Page 36 and 37:
Evaluation of checklists and scales
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
Page 44 and 45:
32TABLE 8 Details of top 60 quality
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
38TABLE 10 Other domains: reporting
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
Use of quality assessment in system
Page 58 and 59:
Page 60 and 61:
Page 62 and 63:
Empirical estimates of bias associa
Page 64 and 65:
Page 66 and 67:
Page 68 and 69:
Page 70 and 71:
Page 72 and 73:
Page 74 and 75:
Page 76 and 77:
Empirical evaluation of the ability
Page 78 and 79:
Page 80 and 81:
Page 82 and 83:
70TABLE 22 Comparison of concurrent
Page 84 and 85:
Page 86 and 87:
74TABLE 26 Comparison of methods of
Page 88 and 89:
Page 90 and 91:
Page 92 and 93:
80TABLE 33 Hypothetical example dem
Page 94 and 95:
82TABLE 34 Hypothetical example dem
Page 96 and 97: Empirical evaluation of the ability
Page 98 and 99: Empirical evaluation of the ability
Page 100 and 101: Discussion and conclusions88histori
Page 102 and 103: Discussion and conclusions90For exa
Page 104 and 105: Discussion and conclusionsNon-rando
Page 107 and 108: Health Technology Assessment 2003;
Page 121: Health Technology Assessment 2003;
Page 124 and 125: Appendix 1data)) or (non-random$ or
Page 126 and 127: Appendix 2AuthorYearENDARESourcePub
Page 128 and 129: Appendix 2Author:Accession No:Endno
Page 130 and 131: Appendix 20 0 00Additional outcomes
Page 132 and 133: Appendix 2Endnote NoWas CMA conside
Page 134 and 135: 122AuthorOrigin aModified toolTool
Page 148 and 149: Appendix 4136DuRant, 1994 99The typ
Page 155 and 156: © Queen’s Printer and Controller
Page 185: Health Technology Assessment 2003;
Page 188 and 189: Health Technology Assessment Progra
Page 190: Health Technology Assessment Progra
show all

Evaluating non-randomised intervention studies - NIHR Health ...

Create successful ePaper yourself

Delete template?

Save as template?