Evaluating non-randomised intervention studies - NIHR Health ...

More documents

Recommendations

Info

82TABLE 34 Hypothetical example demonstrating the potential impact of adjusting for covariates when misclassifications are correlatedObserved unstratified resultsDead Alive TotalT 165 201 366C 143 223 366Total 308 424 732Unadjusted OR = 1.28Dead Alive TotalT 165 201 366C 143 223 366Total 308 424 732Unadjusted OR = 1.28Dead Alive TotalT 165 201 366C 143 223 366Total 308 424 732(a) Results stratified according to underlying distribution of covariates (unobserved) aCV1 = 0, CV2 = 0Dead Alive TotalT 6 24 30C 100 200 300Total 106 224 330OR = 0.5CV1 = 1, CV2 = 0Dead Alive TotalT 30 30 60C 40 20 60Total 70 50 120OR = 0.5True underlying OR = 0.50, OR adjusted only for CV1 (observed without misclassification) = 0.50(b) Results stratified according to covariates with misclassification b in CV1 aCV1 = 0, CV2 = 0Dead Alive TotalT 13 26 39C 82 146 228Total 95 172 267OR = 0.89CV1 = 1, CV2 = 0Dead Alive TotalT 23 28 51C 58 74 132Total 81 102 183OR = 1.05CV1 = 0, CV2 = 1Dead Alive TotalT 6 24 30C 1 2 3Total 7 26 33OR = 0.5CV1 = 0, CV2 = 1Dead Alive TotalT 41 54 95C 1 2 3Total 42 56 98OR = 1.52OR adjusted only for CV1 (observed with misclassification) = 1.17CV1 = 1, CV2 = 1(c) Results stratified according to covariates with additional misclassification c in CV2 for those misclassified on CV1 aCV1 = 0, CV2 = 0Dead Alive TotalT 50 63 113C 83 146 229Total 133 209 342CV1 = 1, CV2 = 0Dead Alive TotalT 21 21 42C 28 14 42Total 49 35 84CV1 = 0, CV2 = 1Dead Alive TotalT 4 17 21C 1 1 2Total 5 18 23Dead Alive TotalT 123 123 246C 2 1 3Total 125 124 249OR = 0.5CV1 = 1, CV2 = 1Dead Alive TotalT 88 93 181C 2 1 3Total 90 94 184OR = 0.47CV1 = 1, CV2 = 1Dead Alive TotalT 90 100 190C 31 62 93Total 121 162 283Empirical evaluation of the ability of case-mix adjustment methodologies to control for selection biasUnadjusted OR = 1.28OR = 1.40OR = 0.5OR = 0.24OR = 1.80OR additionally adjusted for CV2 (observed with correlated misclassification) = 1.32a Adjusted OR obtained from Mantel–Haenszel estimators.b Misclassification in CV1: 30% of CV1=0 misclassified as CV1=1, 30% of CV1=1 misclassified as CV1=0.c Misclassification in CV2: CV2 always classified as CV2=1 if CV1 misclassified as CV1=1; CV2 always classified as CV2=0 if CV1 misclassified as CV1=0.CV, covariate; CV1, first covariate; CV2, second covariate.
Health Technology Assessment 2003; Vol. 7: No. 27partially for the effect of the covariate. The logicalcorollary of this assumption is that it is alwaysworthwhile to adjust for a covariate, howeverpoorly it has been measured. However, this notionin turn depends critically on the assumption thatmisclassification in the covariate is itselfunbiased. 157 Greenland and Robins 162 discussedscenarios for case-control studies where thisassumption is relaxed. They considereddifferential misclassification in disease, exposureand a single covariate, and showed that in somemisclassification scenarios it can actually bedetrimental to adjust for the covariate – theunadjusted estimate may be closer to theunderlying effect than the adjusted estimate. Inthese situations it is not a matter of adjustmentbeing inefficient but rather that adjustment leadsto totally incorrect conclusions. Such an error willbe magnified by the additional credence oftengiven to an adjusted analysis when reporting studyresults.Greenland and Robins’ scenarios hint at amechanism that might explain our troublingfourth key result. However, their scenarios do nottranslate directly to consideration of nonrandomisedintervention studies: they consideredmeasurement error in the exposure (treatment)and case–control status (outcome). They also onlyassessed the impact of measurement error in asingle covariate, whereas case-mix adjustmentinvolves many covariates, all of which could besubject to misclassification. We have extended thehypothetical example in Table 34 to includestratification for a second covariate (CV2). InTable 34(c) we have introduced misclassification inCV2 but only for those misclassified for CV1. Weobserve that the OR adjusted for observed CV1and CV2 (OR = 1.32) is now further from theunderlying true value (OR = 0.5) than theunadjusted estimate (OR = 1.28). Hence such amechanism could explain our fourth observation,that adjustment in some situations can increasebias in estimates of treatment effects.If misclassification was solely a matter ofmeasurement error, such correlated misclassificationswould be rare. However, misclassification includesall processes that lead us to observe a value thatdiffers from the true value of the confounder. Forexample, consider a study in which the trueconfounder is a person’s average blood pressure.The blood pressure reading used in the analysiscould be misclassified owing to measurementerror, but it could also be misclassified owing tonatural within-person variation in blood pressurerelated to the time of day when the measurementwas made, or to a short period of stress. If asecond variable was included in the analysis thatwas also influenced by circadian rhythm or stress,misclassification in the second variable will becorrelated with misclassification in blood pressure.If case-mix adjustment models are beinginfluenced by correlated misclassification, it couldbe hypothesised that including more variables inan adjustment model will increase the chances ofcorrelated misclassifications, and hence increaseexpected bias. Comparison of the three differentlogistic models across Tables 24, 25, 27–29 and32 potentially supports this hypothesis: in nearlyall cases the logistic regression model includingthe most covariates (the ‘full model’) was morebiased than the logistic regression model with thefewest covariates (backward stepwise withp-to-remove of 0.05).Misclassification and measurement error incovariates has been cited as the possible root ofmany controversial associations detected throughobservational studies. 163 We are correspondinglyconcerned that measurement and misclassificationerrors may be largely responsible for our fourthresult.Differences between unconditional andconditional estimatesCovariates which are prognostic but balancedacross groups are rarely routinely adjusted for inanalyses of randomised controlled trials. Evenwhen adjustments are made, the adjustedestimates are often ignored when results of trialsare included in meta-analyses. We have followedthis standard approach in calculating the resultsof the RCTs that we created from the IST andECST data sets. These unadjusted estimates areknown as unconditional or population averageresults.Estimates of treatment effects where prognosticvariables have been adjusted for are known asconditional estimates, being conditional onknowledge of the covariates included in the analysis.Our comparison of results of RCTs with adjustedresults of non-randomised studies is thereforecomparing unconditional estimates of treatmenteffects from RCTs with conditional estimates oftreatment effects from non-randomised studies.When using ORs, it has been noted thatadjustment for prognostic covariates leads todifferences between unconditional and conditionalestimates of treatment effects, even for covariatesthat are balanced across treatment groups. 164,16583© Queen’s Printer and Controller of HMSO 2003. All rights reserved.
Page 1 and 2:
Health Technology Assessment 2003;
Page 3 and 4:
Evaluating non-randomisedinterventi
Page 5 and 6:
Page 7:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17:
Page 21 and 22:
Page 23 and 24:
© Queen’s Printer and Controller
Page 25 and 26:
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33:
Page 36 and 37:
Evaluation of checklists and scales
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
Page 44 and 45: 32TABLE 8 Details of top 60 quality
Page 46 and 47: Evaluation of checklists and scales
Page 50 and 51: 38TABLE 10 Other domains: reporting
Page 56 and 57: Use of quality assessment in system
Page 62 and 63: Empirical estimates of bias associa
Page 76 and 77: Empirical evaluation of the ability
Page 82 and 83: 70TABLE 22 Comparison of concurrent
Page 86 and 87: 74TABLE 26 Comparison of methods of
Page 92 and 93: 80TABLE 33 Hypothetical example dem
Page 100 and 101: Discussion and conclusions88histori
Page 102 and 103: Discussion and conclusions90For exa
Page 104 and 105: Discussion and conclusionsNon-rando
Page 107 and 108: Health Technology Assessment 2003;
Page 121: Health Technology Assessment 2003;
Page 124 and 125: Appendix 1data)) or (non-random$ or
Page 126 and 127: Appendix 2AuthorYearENDARESourcePub
Page 128 and 129: Appendix 2Author:Accession No:Endno
Page 130 and 131: Appendix 20 0 00Additional outcomes
Page 132 and 133: Appendix 2Endnote NoWas CMA conside
Page 134 and 135: 122AuthorOrigin aModified toolTool
Page 144 and 145:
132AuthorOrigin aModified toolTool
Page 146 and 147:
134AuthorOrigin aModified toolTool
Page 148 and 149:
Appendix 4136DuRant, 1994 99The typ
Page 151 and 152:
Page 153 and 154:
Page 155 and 156:
Page 157 and 158:
Page 159 and 160:
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185:
Page 188 and 189:
Health Technology Assessment Progra
Page 190:
Health Technology Assessment Progra
show all

Evaluating non-randomised intervention studies - NIHR Health ...

Create successful ePaper yourself

Delete template?

Save as template?