Evaluating non-randomised intervention studies - NIHR Health ...

Health Technology Assessment 2003; Vol. 7: No. 27TABLE 27 Comparison of methods of case-mix adjustment applied to results of concurrently controlled studies with results of RCTsresampled from 14 regions within the ISTPercentage of studies withstatistically significantAverageVariability of results results (p < 0.05)OR SD of log OR Ratio with RCT Benefit Harm TotalRCTs 0.91 0.34 7 2 9Concurrently controlled studiesUnadjusted 0.91 0.85 2.51 29 21 50Stratification 0.92 0.89 2.70 27 21 48Logistic regressionFull model a 0.92 1.12 3.39 23 18 41Stepwise p r = 0.05 b 0.91 1.01 3.06 26 20 46Stepwise p r = 0.15 c 0.91 1.03 3.12 26 20 46Propensity scoreMatched d 0.95 0.79 2.39 15 12 27Stratified 0.94 0.76 2.30 19 15 34Regression 0.95 0.75 2.27 19 15 34a Full model includes 10 covariates.b Mean number of covariates included: 4.6.c Mean number of covariates included: 5.8.d Mean number of patients matched: 101 out of 200.TABLE 28 Comparison of methods of case-mix adjustment applied to results of concurrently controlled studies with results of RCTsresampled from 10 UK cities within the ISTPercentage of studies withstatistically significantAverageVariability of results results (p < 0.05)OR SD of log OR Ratio with RCT Benefit Harm TotalRCT 1.01 0.49 7 6 13Concurrently controlled studiesUnadjusted 1.02 0.90 1.84 22 23 45Stratification 0.99 0.91 1.86 22 21 43Logistic regressionFull model a 1.05 0.96 1.96 16 17 33Stepwise p r = 0.05 b 1.03 0.89 1.82 18 20 38Stepwise p r = 0.15 c 1.04 0.89 1.82 18 20 38Propensity scoreMatched d 1.05 0.84 1.71 11 13 24Stratified 1.05 0.79 1.61 14 16 30Regression 1.03 0.74 1.51 13 15 28a Full model includes 10 covariates.b Mean number of covariates included: 3.4.c Mean number of covariates included: 4.4.d Mean number of patients matched: 109 out of 200.situations, and a large bias for concurrentlycontrolled studies from the ECST.Adjustment for bias related to ‘known’differences in case-mixIn this section we summarise the results for thethree sets of studies constructed to mimic studieswhere allocation occurred by indication.Adjusting for bias due to a known and measuredcovariateThe results of case-mix adjustment for the ISTstudies with bias relating to neurological deficit aregiven in Table 30. In these analyses systematic biaswas introduced according to values of a singlecovariate (neurological deficit). The samecovariate was then included in the models, and75© Queen’s Printer and Controller of HMSO 2003. All rights reserved.