Evaluating non-randomised intervention studies - NIHR Health ...

Health Technology Assessment 2003; Vol. 7: No. 27Chapter 2Aims and objectivesThis project has considered the methods andevidence-base for evaluating non-randomisedintervention studies. It is restricted to designs 3–6described in Chapter 1 (Box 1), and thus excludescase–control designs. The report first presents thefindings from reviews of current evidence andexisting tools for evaluating non-randomisedstudies, and the application of these tools insystematic reviews. It concludes with reports of twonew empirical investigations of expected biases andmethods for correcting for bias in non-randomisedstudies. The specific objectives addressed in eachof the chapters are presented below.Chapters 3–5 report the results of three separatereviews of the literature.Chapter 3 reviews the current empirical evidenceconcerning bias associated with non-randomisedstudies of healthcare interventions. In particular, itconsiders the evidence:●●●●that there are inconsistencies between the resultsof randomised and non-randomised studiesthat non-randomised studies may besystematically biasedthat the results of non-randomised studies maybe more variable than the results of RCTsthat case-mix adjustment reduces systematicbias and variability in non-randomised studies.Chapter 4 reviews existing tools for assessing thequality of non-randomised studies to:●●●describe the content, development and usabilityof quality assessment tools for non-randomisedstudiesappraise the degree to which the assessmenttools evaluate specified quality domainsidentify the tools most suitable for use insystematic reviews of non-randomised studies,and to identify their limitations.In Chapter 5, the use of quality assessment insystematic reviews is considered. The review aimsto:●evaluate the quality assessments made insystematic reviews that have included nonrandomisedstudies●investigate whether relationships between studyquality and study results have been observed.Chapters 6 and 7 report the results of newempirical investigations obtained through analysisof non-randomised studies generated byresampling participants from two large,multicentre randomised trials. The rationale andmethodology are described in detail in Chapter 6.The investigation reported in Chapter 6evaluates:●●the degree of systematic bias introduced by theuse of historical and concurrent controls in nonrandomisedstudies of healthcare interventionsand the impact that this has on studyconclusionsthe degree to which results of non-randomisedstudies using historical and concurrent controlsare more variable than those of randomisedcontrolled trials and the impact that this has onstudy conclusions.In Chapter 7, the investigation is extended toevaluate the ability of case-mix adjustmentmethods to correct selection bias introduced innon-randomised designs. Specifically it considers:●●●●whether case-mix adjustment methods cancompensate for systematic bias introducedthrough the use of historical and concurrentcontrolswhether case-mix adjustment methods cancompensate for increased variability introducedthrough the use of historical and concurrentcontrolswhether case-mix adjustment methods cancorrect for bias introduced through mechanismsakin to allocation by indicationwhether there are differences in theperformance of stratification, logistic regressionand propensity score methods for adjusting forselection bias.Chapter 8 presents an overview of the findingsfrom all sections of the project and makesrecommendations concerning the evaluation ofnon-randomised studies and further research thatshould be undertaken.7© Queen’s Printer and Controller of HMSO 2003. All rights reserved.