Evaluating non-randomised intervention studies - NIHR Health ...

More documents

Recommendations

Info

Empirical evaluation of the ability of case-mix adjustment methodologies to control for selection biasRegion Trial allocation Number of neurological deficits0 1 2 3 4 56Northern ItalyAspirinAvoid aspirinT 0,1C 0,1T 1,1C 1,1T 2,1C 2,1T 3,1C 3,1T 4,1C 4,1T 5,1C 5,1T 6,1C 6,1Central ItalyAspirinAvoid aspirinT 0,2C 0,2T 1,2C 1,2T 2,2C 2,2T 3,2C 3,2T 4,2C 4,2T 5,2C 5,2T 6,2C 6,2Southern ItalyAspirinAvoid aspirinT 0,3C 0,3T 1,3C 1,3T 2,3C 2,3T 3,3C 3,3T 4,3C 4,3T 5,3C 5,3T 6,3C 6,3ScotlandAspirinAvoid aspirinT 0,4C 0,4T 1,4C 1,4T 2,4C 2,4T 3,4C 3,4T 4,4C 4,4T 5,4C 5,4T 6,4C 6,4Northern England and WalesAspirinAvoid aspirinT 0.5C 0,5T 1.5C 1.5T 2.5C 2.5T 3.5C 3.5T 4.5C 4.5T 5.5C 5.5T 6.5C 6.5Southern EnglandAspirinAvoid aspirinT 0,6C 0,6T 1,6C 1,6T 2,6C 2,6T 3,6C 3,6T 4,6C 4,6T 5,6C 5,6T 6,6C 6,6AustraliaAspirinAvoid aspirinT 0,7C 0,7T 1,7C 1,7T 2,7C 2,7T 3,7C 3,7T 4,7C 4,7T 5,7C 5,7T 6,7C 6,7The NetherlandsAspirinAvoid aspirinT 0,8C 0,8T 1,8C 1,8T 2,8C 2,8T 3,8C 3,8T 4,8C 4,8T 5,8C 5,8T 6,8C 6,8New ZealandAspirinAvoid aspirinT 0,9C 0,9T 1,9C 1,9T 2,9C 2,9T 3,9C 3,9T 4,9C 4,9T 5,9C 5,9T 6,9C 6,9NorwayAspirinAvoid aspirinT 0,10C 0,10T 1,10C 1,10T 2,10C 2,10T 3,10C 3,10T 4,10C 4,10T 5,10C 5,10T 6,10C 6,10PolandAspirinAvoid aspirinT 0,11C 0,11T 1,11C 1,11T 2,11C 2,11T 3,11C 3,11T 4,11C 4,11T 5,11C 5,11T 6,11C 6,11SpainAspirinAvoid aspirinT 0,12C 0,12T 1,12C 1,12T 2,12C 2,12T 3,12C 3,12T 4,12C 4,12T 5,12C 5,12T 6,12C 6,12SwedenAspirinAvoid aspirinT 0,13C 0,13T 1,13C 1,13T 2,13C 2,13T 3,13C 3,13T 4,13C 4,13T 5,13C 5,13T 6,13C 6,13SwitzerlandAspirinAvoid aspirinT 0,14C 0,14T 1,14C 1,14T 2,14C 2,14T 3,14C 3,14T 4,14C 4,14T 5,14C 5,14T 6,14C 6,1466FIGURE 14 Resampling structure for IST based on number of neurological deficits
Health Technology Assessment 2003; Vol. 7: No. 27TABLE 21 Sampling probabilities use to generate non-randomised designs mimicking ‘allocation by indication’(a) Sampling probabilities by number of neurological deficits, IST0 1 2 3 4 5 6Aspirin 1.0 0.9 0.8 0.6 0.4 0.2 0.1Avoid aspirin 0.1 0.2 0.4 0.6 0.8 0.9 1.0(b) Sampling probabilities by consciousness on presentation, ISTFully alertDrowsy or unconsciousAspirin 1.0 0.5Avoid aspirin 0.5 1.0(c) Sampling probabilities by outcome, ISTAdverse outcomeNo adverse outcomeExperimental 0.75 1.00Control 1.00 0.75which patients have been allocated interventionsbased on their characteristics. These additionalanalyses were carried out on the IST data only.Non-randomised studies mimicking ‘allocation byindication’ were generated by identifying strongprognostic covariates in the IST and biasingallocation according to values of these covariates.These studies were based on a simplistic form ofallocation by indication, in which the decision totreat depends on a single observed covariate. Twodifferent covariates were considered separately:the number of neurological deficits and the levelof consciousness at admission. The prognosticvalue of these covariates is summarised inAppendix 8.To generate bias, participants within each regionwere divided into groups according to their valuesof the covariate and the treatment that theyreceived. For example, Figure 14 shows thecategorisation of participants in the IST accordingto whether they receive aspirin (T) or avoid aspirin(C), the number of neurological deficits with whichthey present (0–6, first subscript) and the regionwithin which they were recruited (1–14, secondsubscript). Participants were resampled from thisstructure with sampling probabilities determinedaccording to their allocation and covariate value.For both covariates sampling probabilities werechosen such that the treated group was weightedtowards participants with good prognosis, whereasthe control group was weighted towardsparticipants with poor prognosis, leading to anoverestimate of the benefit of experimentaltreatment. The sampling probabilities used withthe two covariates are given in Table 21(a) and (b).The same sampling probabilities were used in allregions.In practice, allocation by indication is morecomplicated than the single covariate allocationdescribed above – it depends on some unknowncombination of multiple pieces of covariateinformation that are predictive of prognosis, andis likely to vary both between individual cliniciansand over time. We wished to include studies withsuch a complex bias in our evaluation toinvestigate how well case-mix adjustment methodsmight work in more typical database-type analyses.Research into medical decision making hascharacterised some decision processes usingregression techniques to provide modelsexplaining how covariate information is combinedto make treatment decisions. Although it would bepossible to generate biased allocations using sucha decision model, we did not use this approach asit is too simplistic and artificial to test properly theability of case-mix adjustment methods to adjustfor selection bias. Such analyses wouldoverestimate the ability of case-mix adjustmentmethods to adjust for bias due to the circularity ofone regression model (the case-mix adjustmentmethod) being used to estimate parameters of anunderlying regression model (the allocationprocess). The many treatment decisions that didnot fit with the underlying regression model wouldnot be accounted for.The second approach that we used is based on theobservation that when decisions are made inrelation to prognosis, those decisions naturally67© Queen’s Printer and Controller of HMSO 2003. All rights reserved.
Page 1 and 2:
Health Technology Assessment 2003;
Page 3 and 4:
Evaluating non-randomisedinterventi
Page 5 and 6:
Page 7:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17:
Page 21 and 22:
Page 23 and 24:
© Queen’s Printer and Controller
Page 25 and 26:
Page 27 and 28: © Queen’s Printer and Controller
Page 29 and 30: Health Technology Assessment 2003;
Page 33: Health Technology Assessment 2003;
Page 36 and 37: Evaluation of checklists and scales
Page 44 and 45: 32TABLE 8 Details of top 60 quality
Page 50 and 51: 38TABLE 10 Other domains: reporting
Page 56 and 57: Use of quality assessment in system
Page 62 and 63: Empirical estimates of bias associa
Page 76 and 77: Empirical evaluation of the ability
Page 82 and 83: 70TABLE 22 Comparison of concurrent
Page 86 and 87: 74TABLE 26 Comparison of methods of
Page 92 and 93: 80TABLE 33 Hypothetical example dem
Page 94 and 95: 82TABLE 34 Hypothetical example dem
Page 100 and 101: Discussion and conclusions88histori
Page 102 and 103: Discussion and conclusions90For exa
Page 104 and 105: Discussion and conclusionsNon-rando
Page 121: Health Technology Assessment 2003;
Page 124 and 125: Appendix 1data)) or (non-random$ or
Page 126 and 127: Appendix 2AuthorYearENDARESourcePub
Page 128 and 129:
Appendix 2Author:Accession No:Endno
Page 130 and 131:
Appendix 20 0 00Additional outcomes
Page 132 and 133:
Appendix 2Endnote NoWas CMA conside
Page 134 and 135:
122AuthorOrigin aModified toolTool
Page 136 and 137:
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Appendix 4136DuRant, 1994 99The typ
Page 151 and 152:
Page 153 and 154:
Page 155 and 156:
Page 157 and 158:
Page 159 and 160:
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185:
Page 188 and 189:
Health Technology Assessment Progra
Page 190:
Health Technology Assessment Progra
show all

Evaluating non-randomised intervention studies - NIHR Health ...

Create successful ePaper yourself

Delete template?

Save as template?