Mental health policy and practice across Europe: an overview

The evidence base 113
Learning from evaluation of the Care Programme Approach
Past experience can also help us to think carefully about the design of future
studies so that they can address questions that policy-makers want to answer.
It also emphasizes the importance of careful interpretation of study findings
and using caution when drawing conclusions. Again, looking at the United
Kingdom, the Care Programme Approach (CPA) is an example of a policy that
was introduced in response to public (and hence political) pressure following a
clinical incident. It was also introduced in the light of other countries’ innovative
developments around case management, but was not a direct transportation
of other country models; rather this was an approach unique to the United
Kingdom. The programme was not based on a prior CPA study, nor was it the
same model as that used in case management trials which were subsequently
conducted by a number of research teams in the United Kingdom. The key
components of the CPA were: the existence of a key worker rather than a case
manager; assessment; a care plan; and regular review as necessary.
Case management trials conducted, and subsequently reviewed in a Cochrane
review (Marshall et al. 1998), were, in fact, about a case management model
in which a specific case manager (not a key worker) was assigned to a client.
Furthermore, the United Kingdom trials of case management were conducted
after the introduction of CPA, and so the control populations were, at least
theoretically, (also) in receipt of CPA. The outcome variables chosen for evaluation
in the Cochrane review included hospital readmissions and it was assumed
that this was an undesirable outcome. Yet, it was just as likely that increased
attention to clients led to less neglect; thus for many, if the need for a hospital
admission arose, this may have been an appropriate event.
So for all these reasons, the United Kingdom case management trials were not
in any sense an evaluation of CPA as used in the United Kingdom, but that was
exactly what the Cochrane review claimed them to be. This is a good example
of how even rigorous reviews may be flawed by subjective bias. In retrospect,
it is a helpful exercise to consider whether trials should have been conducted
on the separate and joint components of the CPA, or whether it was reasonable
for policy-makers to assume that the key components were sufficiently well
understood to be of clinical value.
Making use of multi-level approaches to evaluation
One way in which the strength of the evidence base for policy-making may be
improved is to make greater use of evaluation designs such as cluster randomized
trials. It is vital to understand the need for multi-level approaches to the
design and interpretation of such evaluations of policy changes. In most, if not
all, examples one is interested in at least two levels: the effect on individuals (or
on households and families) and the effect on a higher-level unit which might
be schools, communities, institutions such as prisons and so on. Policy may be
implemented at the higher-level unit while the aim may well be to achieve a
benefit at the individual level. Clustering is the phenomenon whereby the effect
on individuals tends to be similar among individuals who are embedded within
a higher-level unit. Thus, outcomes for children will be affected in part by the