Evaluating Patient-Based Outcome Measures - NIHR Health ...

The third section of this review examines the
ways in which patient-based outcome measures
need to be assessed. In summary, it is argued that
there are eight dimensions in terms of which a
patient-based measure can be examined. Evidence
about a measure that is being considered for
inclusion in a trial needs to be considered in terms
of the following issues: appropriateness, reliability,
validity, responsiveness, precision, interpretability,
acceptability and feasibility. As has already been
pointed out, these criteria are not rank ordered
in terms of importance and do not follow any
sequential logic in terms of how they should be
approached. For each of the criteria, the evidence
and nature of current views is summarised in order
to make clearer to the reader what is meant by a
criterion. Three of the criteria, appropriateness,
precision and interpretability, have increasingly
been discussed in the literature but are less likely to
appear on check-lists in many standard discussions.
For these criteria, although it is clear from the
literature that they are important, there is no
uniform language or framework in terms of which
they are discussed. The remaining criteria; reliability,
validity, responsiveness, acceptability and
feasibility are more often cited on standard lists
and discussions. In the case of reliability, validity
and responsiveness, this in part reflects their widespread
usage in the field of psychometrics. For
none of the criteria are there absolutely explicitly
defined and universally accepted understandings of
the terms; in many areas there remain uncertainties
and differences of view. The purpose of this section
is steer the reader through current debates about
the eight criteria in a helpful way.
Appropriateness
Is the content of the instrument appropriate to
the questions which the clinical trial is intended
to address?
The first and most fundamental consideration
to be faced when selecting a patient-based outcome
measure is how to identify one that is most
appropriate to the aims of the particular trial.
This requires careful consideration of the aims
Chapter 3
Health Technology Assessment 1998; Vol. 2: No. 14
Criteria for selecting a patient-based
outcome measure
of the trial, with reference to the QoL research
questions, i.e. which dimensions will be primary
and secondary end points, the nature of the study
intervention and of the patient group and about
the content of possible candidate instruments. For
this reason, it is particularly difficult to give specific
recommendations about what in general makes an
outcome measure appropriate to a trial, because
this is ultimately a judgement of the fit between
investigators’ specific trial questions and content
of instruments. However, it is clear from a number
of reviews already carried out in this field that it is
an absolutely fundamental issue.
There have been several previous reviews that
have discussed appropriateness of outcome
measures in clinical trials in general terms. Some
of the early reviews are mainly critical of clinical
trials for failing to use any kind of patient-based
outcome measure where the subject matter seemed
to indicate that such an approach was needed.
Thus Brook and Kamberg (1987) conducted a
MEDLINE review of clinical trials and concluded
that, from a sample of 73 clinical trials in which
they considered health status or QoL was likely to
be a major issue, in only two trials was an appropriate
patient-based outcome measure used.
Najman and Levine (1981) reached the same
conclusions from a range of trials in an earlier
review. A third review finds evidence of trialists
failing to use appropriate outcome measures
even where title, keywords or abstract include
‘quality of life’. Schumacher et al. (1991) reviewed
67 such trials in the fields of oncology and cardiology
and found that 43% of studies included no
serious assessment of QoL at all and a further
24% assessed a limited single aspect that the
reviewers considered inadequate.
A more formal evaluation of outcome measurement
in trials is reported by Guyatt and colleagues
(1989b). In their study, two raters independently
examined all clinical trials published in a range
of journals in 1986. Of the 75 trials they evaluated,
they considered QoL as crucial or important in
55 (73%) of trials. However, in 44% of this subgroup
of trials, no effort was made to assess this
dimension of outcome. In a further 38% of
19