In Search of Evidence

Systematic Reviews and Meta-Analyses
about the way in which the effect has occurred frequently comes from
qualitative research (Petticrew & Roberts, 2003). When assessing the quality of
a systematic review, it will therefore be necessary to check whether the
appropriate types of research have been identified for the research question.
Table 2 gives a summary of the suitability of different types of research with
reference to a specific research question.
Assessing the quality of a systematic review
In order to define the quality of a systematic review, it must be critically
appraised. This can be done with the aid of a number of essential questions
based on Oxman and Guyat's validated index (see Table 3). These questions are
addressed below.
Question 1: Is the review based on an explicit question?
A systematic review must be based on an explicit, adequately focused
question. The medical world makes use of a handy mnemonic for this. Doctors
and medical students have learned to carry out a so-called PICO before starting
to look for evidence. In this acronym the P stands for Patient, Problem or
Population, the I for Intervention, the C for Comparison, and the O for
Outcome. The social sciences have adopted this mnemonic and have added C
for Context (PICOC). The underlying thought is that all five elements are
important in a focused search for evidence, and that each change in the P, I, C,
O or C leads to different evidence and therefore also to a different outcome.
For example, a search based on a question such as “Is participative goal setting
an effective intervention in the event of organizational change?” leads to an
outcome of limited practical value because only the I (participative goal setting)
is addressed in the question, without taking account of
! the P: the effect may be different in a division with a cultural problem
than in an organisation with a structural problem
! the C: the effect may be different in comparison with ‘appreciative
inquiry’ than in comparison with ‘survey feedback’
! the O: the effect on production may be different from the effect on
the readiness of middle managers to change
! and the C: the effect may be different for a financial services provider
than for a teaching hospital.