Mental health policy and practice across Europe: an overview

The evidence base 103
campaigns, it may be difficult for a control group not to be ‘contaminated’ by
information they might receive from the individuals enrolled in the intervention
group. Different approaches to evaluation are required which, from the perspective
of the hierarchy of evidence-based medicine, may be viewed as inferior.
Yet some contend that at least as much value ought to be placed on quantitative
observational studies (Black 1996) or on non-controlled qualitative analyses
(Newburn 2001). Not only can such approaches be used to explore hypotheses
to help in the framing of future quantitative studies, but supporters of qualitative
techniques also argue that these methods better reflect the real-world environment
within which decision-makers operate, and offer a better understanding
of the underlying social mechanisms that cause interventions to succeed or fail
(Sanderson 2000).
But what do we mean by qualitative research? This is a broad term that covers
a wide range of methods which ‘involve the systematic collection, organization,
and interpretation of textual material derived from talk or observation. It is
used in the exploration of meanings of social phenomena as experienced by
individuals themselves, in their natural context’ (Malterud 2001: 483). Unlike
quantitative studies the aim is not to generate statistically significant results, but
to explore themes, patterns and associations within a richer and more diverse
data set. As such, qualitative research is useful for addressing a different set of
evaluation questions: not ‘how many xs?’, but ‘what is x, how does x vary in
different situations and why?’ Examples of the approach can include focus
groups, the observation of organizational settings and team behaviour, and
in-depth interviews.
Such research can help inform the debate on the generalization of findings
from studies. It may be used in the assessment of complex outcomes that are
difficult to quantify (e.g. changes in organizational culture or sense of community),
but which impact on the success of an intervention. They can also be
used to identify differences in the importance of outcomes between different
stakeholders (Pope et al. 2002). One recent systematic review looking at the
effectiveness of electroconvulsive therapy (ECT) indicated that service user-led
research reported significantly less benefit than clinician-led studies. This, it
was argued, may be due to clinical studies obtaining information from those
undergoing ECT too soon after treatment and using simplistic questionnaires
that did not pick up complex patient views. The study highlighted the need for
more qualitative research to identify outcomes of value to those that undergo
treatment (Rose et al. 2003).
Qualitative approaches should not be viewed as an alternative to quantitative
approaches; in fact, they can be seen as complementary. The hierarchy of evidence
seen in EBM, which places least value on qualitative research methods (if
they are mentioned at all), can be replaced by a matrix which emphasizes the
need to match the research design to the research question, as illustrated in
Table 5.1 (Muir-Gray 1996; Petticrew and Roberts 2003). While it is clear that
the systematic pooling of the findings of many studies generally provides the
most robust source of evidence, and that trials best answer questions of efficacy,
effectiveness and cost-effectiveness, the matrix indicates that for questions
of appropriateness, satisfaction, service delivery processes and acceptability,
qualitative research methods provide appropriate research designs.