Evaluating Patient-Based Outcome Measures - NIHR Health ...

20
Criteria for selecting a patient-based outcome measure
the 55 trials, an untested measure was used that
the reviewers considered inappropriate. They
concluded that appropriate measures would have
considerably strengthened the basis for recommendations
emerging from the trial.
Fundamental to such critiques is the view that
measures of outcome used in trials and intended
to assess the patient’s perspective are often limited
or superficial. The strongest expression of this
criticism can be found in a recent review by Gill
and Feinstein (1994). They reviewed 75 articles
selected randomly from medical journals that
include QoL measurement in the context of
health care research. They rated the use of QoL
measures in studies on a range of explicit criteria
with overall scores for articles ranging from 0 to
100. Only 11% of articles achieved scores of 50
or more (i.e. ‘satisfactory’ by at least half their
criteria). They were particularly critical that in
85% of articles, authors had not defined QoL for
the purpose of the study and in 52% of articles had
not explained or justified their selection of QoL
measure. They also considered unsatisfactory the
fact that in 83% of studies, patients were not
invited, in addition to other questions, to respond
to a global overall rating of their QoL. Their
overview reveals quite stringent criteria for what
constitutes an appropriate patient-based outcome
measure. Guyatt and Cook (1994) re-examined
15 randomly selected papers from the sample
reviewed by Gill and Feinstein according to their
own criteria. By their criteria, only 33% failed to
use questionnaire items reflecting matters of
importance to patients and only 27% of studies
used measures that omitted important items.
It is apparent from the much more favourable
ratings by Guyatt and Cook that they employed
different criteria to judge appropriateness of
outcomes. This is clearly seen if the full set of
criteria of the two reviews are examined in full
(Box 4).
Both reviews clearly emphasise the need for
appropriate QoL measures to incorporate
questionnaire items that clearly matter to patients.
However they also differ in important respects.
Gill and Feinstein (1994) argue for the need for
measures that are based on an explicit definition
of QoL, that provide a single global score, that
allow patients to state the relative importance of
issues, that allow patients to give supplementary
answers not included in the questionnaire and
to globally rate QoL and health-related QoL.
Underlying their arguments is a view that instruments
have been too dominated by what they
term ‘psychometric’ principles of reliability
BOX 4 Two competing conceptions of requirements
for judging appropriateness of outcome measures
1. Is QoL conceptually defined?
2. Are domains intended to be measured explicitly
stated?
3. Are selected outcome measures explained or
justified?
4. Are scores aggregated into a single overall score?
5. Are patients able to offer a separate global rating of
QoL?
6. Is a distinction made between overall versus healthrelated
QoL?
7. Are patients able to add supplemental comments?
8. Are patients able to rank the importance of
individual items?
(Adapted from Gill and Feinstein, 1994)
1. Do the authors show that aspects of patients’ lives
measured are important to patients?
2. Have previous studies demonstrated their
importance?
3. Do investigators examine aspects of patients’ lives
that, from clinical experience, it is known that
patients value?
4. Are there aspects of health-related quality of
life that are important to patients but have
been omitted?
5. Are individual patients asked directly to place a
value on their lives?
6. Are instruments used demonstrated to have
reliability, validity and responsiveness?
7. Do instruments used have interpretability (i.e.
distinguish trivial from important differences)?
(Adapted from Guyatt and Cook, 1994)
and validity and insufficient attention has been
given to clinical ‘face validity’, which is largely
established by judgements and statements made
by patients unconstrained by the format of fixed
questionnaire items.
By contrast, Guyatt and Cook, whilst accepting
that the primary focus of instruments should be
on matters of importance to patients, argue that
Gill and Feinstein’s requirements are too stringent.
They appear to argue that Gill and Feinstein place
too much emphasis upon eliciting the values and
priorities within a given study and insufficient
attention to use of established instruments in which
prior development and use have identified matters
of importance. At the heart of this dispute are two
different ways of ensuring that patients’ preferences
and concerns are fully incorporated into
trial study design, by extensive use of global and