Evaluating non-randomised intervention studies - NIHR Health ...

More documents

Recommendations

Info

IntroductionBOX 1 Taxonomy of study designs to assess the effectiveness of an interventionExperimental designsA study in which the investigator has control over at least some study conditions, particularly decisions concerning theallocation of participants to different intervention groups.1. Randomised controlled trialParticipants are randomly allocated to intervention or control groups and followed up over time to assess any differencesin outcome rates. Randomisation with allocation concealment ensures that on average known and unknowndeterminants of outcome are evenly distributed between groups.2. Quasi-randomised trialParticipants are allocated to intervention or control groups by the investigator, but the method of allocation falls short ofgenuine randomisation and allocation concealment (e.g. allocated by date of birth, hospital record number, etc.)3. Non-randomised trial/quasi-experimental studyThe investigator has control over the allocation of participants to groups, but does not attempt randomisation (e.g.patient or physician preference). Differs from a ‘cohort study’ in that the intention is experimental rather thanobservational.Observational designsA study in which natural variation in interventions (or exposure) among study participants is investigated to explore the effectof the interventions (or exposure) on health outcomes.4. Controlled before-and-after studyA follow-up study of participants who have received an intervention and those who have not, measuring the outcomevariable both at baseline and after the intervention period, comparing either final values if the groups are comparable atbaseline, or change scores. It can also be considered an experimental design if the investigator has control over, or candeliberately manipulate, the introduction of the intervention.5. Concurrent cohort studyA follow-up study that compares outcomes between participants who have received an intervention and those who havenot. Participants are studied during the same (concurrent) period either prospectively or, more commonly,retrospectively.6. Historical cohort studyA variation on the traditional cohort study where the outcome from a new intervention is established for participantsstudied in one period and compared with those who did not receive the intervention in a previous period, i.e.participants are not studied concurrently.7. Case–control studyParticipants with and without a given outcome are identified (cases and controls respectively) and exposure to a givenintervention(s) between the two groups compared.8. Before-and-after studyComparison of outcomes from study participants before and after an intervention is introduced. The before and aftermeasurements may be made in the same participants, or in different samples. It can also be considered an experimentaldesign if the investigator has control over, or can deliberately manipulate, the introduction of the intervention.9. Cross-sectional studyExamination of the relationship between disease and other variables of interest as they exist in a defined population atone particular time point.10. Case seriesDescription of a number of cases of an intervention and outcome (no comparison with a control group).Adapted from CRD Report 4. 1752of participants to intervention groups either usingrandomisation of participants, or haphazardallocation by alternation, dates of birth, day of theweek or case record numbers. The key to a wellcontrolledexperimental study is the concealmentof the allocation schedule from the studyinvestigators, such that the allocation of any givenparticipant cannot be predicted. 6 When this is notensured, major bias may be introduced. Inobservational studies, on the other hand, thegroups that are compared are generated accordingto variation in the use of interventions that occursregardless of the study. When allocation isdetermined largely by health professionals, thetreatment decision is based not only on ‘hard’ datasuch as age, sex and diagnostic test results, butalso on ‘soft’ data, including type and severity ofsymptoms, rate of development of the illness, andseverity of any co-morbid conditions, which arerarely made explicit. 7 Allocation in observational
Health Technology Assessment 2003; Vol. 7: No. 27TABLE 1 Sources of biasSource of bias RCTs Cohort studiesSelection bias Randomisation Control for confoundersPerformance bias Blinding (of participants and/or investigators) Measurement of exposureAttrition bias Completeness of follow-up Completeness of follow-upDetection bias Blinded outcome assessment Blinded outcome assessmentstudies may also be based on factors such asavailability of care or geographical location. Inobservational studies, therefore, there are likely tobe systematic differences in the case-mix ofpatients allocated to the intervention andcomparison groups.Allocation to groups can also be based on patientchoice, as in patient preference trials. 8 Individualpreferences for one treatment above another maywell imply differences in other consistent ways(potentially relating to prognosis), from those whodo not hold such a preference. 9 In addition,preference for a particular treatment may enhanceits therapeutic effect.Sources of bias in nonrandomisedstudiesThe Cochrane Collaboration handbook has laidout the four main sources of systematic bias intrials of the effects of healthcare as being selectionbias, performance bias, attrition bias and detectionbias (Table 1). All of these biases can affect nonrandomisedstudies and are discussed in thisreport. However, it is selection bias that we discussin the greatest detail, and evaluate in newempirical studies, as it is the potential for selectionbias that most clearly differentiates randomisedfrom non-randomised studies.Selection biasThe greatest distinction between the results ofrandomised and non-randomised studies is, asdescribed above, the risk of selection bias, wheresystematic differences in comparison groups ariseat baseline. The term selection bias can bemisleading as it is used to describe both biasedselection of participants for inclusion in a study(which applies to both experimental andobservational studies) and biased allocation ofpatients to a given intervention (which occurswhere randomisation is not used). The first type ofselection bias is usually classified as an issue ofexternal validity and is not discussed further inthis report. Rather, we consider the second type,which is an issue of internal validity. It is sometimesreferred to as case-mix bias, or confounding.In non-randomised studies, selection bias will beintroduced when participants chosen for oneintervention have different characteristics fromthose allocated to the alternative intervention (ornot treated). For observational studies the choiceof a given intervention is largely at the discretionof the treating clinician (as would occur in normalclinical practice). The choice of an interventionunder these circumstances will be influenced notonly by a clinician’s own personal preference forone intervention over another but also by patientpreference, patient characteristics and clinicalhistory. Sometimes the reasons for the choice of atreatment will be obvious, but at other times aclinician’s treatment decision will be influenced bysubtle clues that are not easily identifiable. 3 Thismay result in treatment groups that areincomparable, often with one intervention group‘heavily weighted by the more severely ill’. 10According to Miettinen, 11 in clinical practice (andtherefore in observational studies):“Interventions are commonly prompted by anindication, a state or event that signifies the prospectof an untoward outcome. Thus, by the very rationalityof decisions to intervene, the treated tend to differfrom the untreated with respect to their outlooks forthe outcome criterion in efficacy assessment; theretends to be confounding by the indication – usually suchthat the treated tend to have less favourable outcomethan the untreated.”In other words, when faced with a patient whomay be eligible to receive a given intervention,the decision to treat will be influenced by somefactor that in turn is related to the treatmentoutcome. This introduces systematic bias leadingto either over- or underestimates of treatmenteffects, depending on the treatment decisionmechanism.Confounding by indication can take several guisesand the term has been used to describe a numberof situations. The original definition refers to thesituation where an extraneous condition is both a3© Queen’s Printer and Controller of HMSO 2003. All rights reserved.
Page 1 and 2: Health Technology Assessment 2003;
Page 3 and 4: Evaluating non-randomisedinterventi
Page 7: Health Technology Assessment 2003;
Page 23 and 24: © Queen’s Printer and Controller
Page 27 and 28: © Queen’s Printer and Controller
Page 36 and 37: Evaluation of checklists and scales
Page 44 and 45: 32TABLE 8 Details of top 60 quality
Page 50 and 51: 38TABLE 10 Other domains: reporting
Page 56 and 57: Use of quality assessment in system
Page 62 and 63: Empirical estimates of bias associa
Page 64 and 65:
Empirical estimates of bias associa
Page 66 and 67:
Page 68 and 69:
Page 70 and 71:
Page 72 and 73:
Page 74 and 75:
Page 76 and 77:
Empirical evaluation of the ability
Page 78 and 79:
Page 80 and 81:
Page 82 and 83:
70TABLE 22 Comparison of concurrent
Page 84 and 85:
Page 86 and 87:
74TABLE 26 Comparison of methods of
Page 88 and 89:
Page 90 and 91:
Page 92 and 93:
80TABLE 33 Hypothetical example dem
Page 94 and 95:
82TABLE 34 Hypothetical example dem
Page 96 and 97:
Page 98 and 99:
Page 100 and 101:
Discussion and conclusions88histori
Page 102 and 103:
Discussion and conclusions90For exa
Page 104 and 105:
Discussion and conclusionsNon-rando
Page 107 and 108:
Health Technology Assessment 2003;
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
Page 117 and 118:
Page 119 and 120:
Page 121:
Page 124 and 125:
Appendix 1data)) or (non-random$ or
Page 126 and 127:
Appendix 2AuthorYearENDARESourcePub
Page 128 and 129:
Appendix 2Author:Accession No:Endno
Page 130 and 131:
Appendix 20 0 00Additional outcomes
Page 132 and 133:
Appendix 2Endnote NoWas CMA conside
Page 134 and 135:
122AuthorOrigin aModified toolTool
Page 136 and 137:
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Appendix 4136DuRant, 1994 99The typ
Page 151 and 152:
Page 153 and 154:
Page 155 and 156:
© Queen’s Printer and Controller
Page 157 and 158:
Page 159 and 160:
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185:
Page 188 and 189:
Health Technology Assessment Progra
Page 190:
Health Technology Assessment Progra
show all

Evaluating non-randomised intervention studies - NIHR Health ...

Create successful ePaper yourself

Delete template?

Save as template?