Evaluating non-randomised intervention studies - NIHR Health ...

160Review Method of incorporating Results of quality investigationquality into synthesisKleijnen, 1991 69 Qualitative Concentrated on results of trials with the best methodological quality. Evidence found to be largely positive,especially in trials of lower qualityKrause, 1998 306 Qualitative Only higher scoring studies were included. A ‘best’ estimate of effectiveness was derived from highest rankingstudies, then a range of probable estimates of effectiveness was derived from the other studies. Authors concludethat there is a suggestion of effectiveness, but methodological limitations make stronger conclusions difficultKreulen, 1998 187 Qualitative Methodological characteristics were discussed narratively; authors conclude that studies were well designed butthat methodological characteristics could be improvedKrywanio, 1994 363 Quantitative:various No significant correlation between ES and quality score regardless of mean birthweight, i.e. the interventionCorrelation analysis of ES and quality score. groups in high-quality studies did not gain more/less than those in lower quality studies.Weighting by sample size, quality score and In one mean birth weight subgroup, only weighting by quality score had an impact on ES [reduced from 1.54mean birth weight in turn(95% CI: 1.22 to 1.86) to 1.37]. Authors comment that this is indicative of poorer quality studies with larger ES.For the other subgroup, only weighting by sample size impacted on ES [reduced from 0.51 (95% CI: 0.39 to0.63) to 0.42], indicating that studies in this group were of higher qualityKulik, 1990 308 Quantitative Random assignment and teacher effects (same or different teachers teaching experimental and control groups)Subgroup analyses according to six quality made little difference to ES or SE. Use of national standardised tests produced lower ESs than locally developedcriteria, further subdivided by teaching tests. Subjectively scored criterion tests produced lower ES in the PSI group compared with objective, machineapproach,i.e. ‘Keller’s Personalised System scoreable exams, but made little difference to the LFM group. Where control groups received the same amountof Instruction’ (largely self-directed learning), of feedback as the intervention groups, the effect size was found to be smaller than where they receivedor Bloom’s ‘Learning for Mastery’standard feedback(teacher-presented and controlled)Kwakkel, 1997 188 Qualitative Methodological limitations of studies were mentioned, and considered as major confounding factors in theconclusionsLee, 1994 312 Qualitative Methodological characteristics discussed separately from results of individual studies; few high-quality studieswere identified; however, some benefits from the interventions were reportedLee, 1997 189 Not consideredLijesen, 1995 303 Qualitative NRS all scored