Full Report - Research for Development - Department for ...

second stage previously described for electronic databases. References from
handsearching were compared with records stored in the first EndNote bibliographic
database and screened for inclusion, as for references from websites searches.
2.2.4 Characterising included studies
We first created and entered into EPPI-Reviewer 4 a coding tool to extract and
record information on generalities, context, intervention, study design, outcomes,
findings, comments, quality assessment and meta-analysis inclusion (Appendix 2.5).
2.2.5 Identifying and describing studies: quality assurance process
Two reviewers screened the results of searches independently. Before starting
screening, a cross-check on a sample of 100 references was conducted to test for
inter-reviewer variability, and where there was disagreement on inclusion, reviewers
consulted with each other until consensus was reached. When consensus could not
be reached, a decision was made after discussion by the team as a whole.
Included studies stored in EPPI-Reviewer4 were randomly allocated to reviewers for
coding. Reviewers coded the results of searches independently. A cross-check on
coding was conducted to test for inter-reviewer variability, and where there was
disagreement on inclusion, reviewers consulted with each other until consensus was
reached. When consensus could not be reached, a decision was made after
discussion within the team.
2.3 Methods for synthesis
2.3.1 Assessing the quality of studies
We reviewed the quality for quantitative and qualitative studies making use of new
revised guidelines for both quantitative and qualitative studies published by the
National Institute for Health and Clinical Excellence’s Centre for Public Health
(2009). In the case of quantitative studies, these checklists (Appendix 2.6) allowed
us to separately make an assessment of internal and external validity using objective
criteria. In terms of external validity, we assessed whether the source population or
study area was well described, whether the eligible study population was
representative of the study population or study area and finally whether the actual
participants were representative of the study population. We evaluated, for
example, whether studies used theoretical perspectives, had clear aims and
discussed the limitations and possible biases. Most of the evaluation of the methods
concerned: whether the randomised controlled trial or the baseline of quantitative
or review methodologies were laid out clearly, for instance, the methods of
randomisation, participant blinding, sample size based on power calculations to
detect significant effects; and whether the conclusions were appropriately justified
by the methods and data. If most of these criteria were fulfilled, then the analysis
would be judged to have a high degree of internal validity (++); if some of the
criteria had been fulfilled, then it would be coded as (+), whilst those studies where
few criteria were met would be coded as (-).
In the case of qualitative studies, the focus was on whether a theoretical approach
could be identified, and whether a clear explanation and justification for the study
design, and information on data collection methods, contexts and discussion of
conclusions and findings were provided, using NICE (2009) guidelines (Appendix 2.7).
Studies would be rated as (++) if they fulfilled most criteria appropriately, (+) for
some criteria and (-) if few criteria were met. In addition, we made use of a
combination of criteria from two different economic evaluation checklists for
economic studies conducted prospectively or retrospectively alongside effectiveness
studies (Drummond and Jefferson, 1996; Evers et al., 2005). A separate checklist
Methods
What are the effects of different models of delivery for improving maternal and infant health
outcomes for poor people in urban areas in low income and lower middle income countries? 19