Evaluating Patient-Based Outcome Measures - NIHR Health ...

42
Criteria for selecting a patient-based outcome measure
instruments should be reduced in terms of
length and number of items in order to increase
acceptability (Burisch, 1984). Some comparative
studies have shown no loss of responsiveness when
such shorter instruments are used (Fitzpatrick
et al., 1989; Katz et al., 1992). Thus, the SF-12 has
emerged as a shorter version of the SF-36, considered
to require only 2 minutes or less to complete
whilst reproducing more than 90% of the
variance in SF-36 scores in the general population
(Ware et al., 1996). Similarly the SIP has undergone
a number of attempts to reduce it from its full
136-item version (McDowell and Newell, 1996).
Whilst there are good reasons in particular
circumstances to prefer shortened versions of
instruments, attention is needed to how an instrument
has been shortened. A recent structured
review of 42 studies intended to shorten longer
original measures found that the majority used
statistical methods alone to achieve this objective,
typically relying on correlations of shorter with
longer versions in the same data, or methods to
maximise internal consistency of the shorter
version (Cronbach’s alpha) (Coste et al., 1997).
The authors argue that, whilst there are obvious
advantages in shortening a well developed and
widely validated longer instrument, there are also
methodological pitfalls. In particular, selection of
items on the basis of internal consistency will
further narrow the scale. They argue that the
psychometric properties of the short version
need to be examined as if it is a new instrument.
Properties such as precision, as discussed in an
earlier section of this review may also be jeopardised
by an instrument with fewer items.
Direct assessment of acceptability
It is preferable directly to assess patients’ views
about a new questionnaire. Sprangers and
colleagues (1993) argue that patients’ views
should be obtained at the pre-testing phase
prior to formal tests for reliability etc., by means
of a structured interview in which they are asked
whether they found any questionnaire items
difficult annoying or distressing or whether issues
were omitted. When the EORTC QLQ-C30 was
assessed in this way, 10% of patients reported that
one or more items were confusing or difficult to
answer and less than 3% that an item was upsetting,
whilst more generally patients welcomed the opportunity
to report their experiences (Aaronson et al.,
1993). Another formal evaluation of acceptability
of a questionnaire found 89% enjoyed the task
of completing the COOP instrument and 97%
reported understanding the questions (Nelson
et al., 1990).
Weinberger and colleagues (1996) directly asked
patients for their preferences for different forms
of administration of the SF-36. Far more positive
preference was expressed for face-to-face interview
compared with either self complete or telephone
based administration.
Not all studies of patient acceptability of instruments
are positive. In a qualitative study of
patients’ views of a QoL assessment that included
an early form of EORTC questionnaire, patients
complained about length, difficulties in understanding
the format of the questionnaire and
possible risks that their answers would influence
subsequent treatment decisions (Bernhard
et al., 1995).
In general, users should expect to see evidence
of acceptability being examined at the design
stage. Subsequently, the most direct and easy to
assess evidence is the length and response rates
of questionnaires.
Translation and cultural applicability
One basic way in which a questionnaire may fail
to be acceptable is if it is expressed in a language
unfamiliar to respondents. This issue has received
a large amount of attention in recent literature
on patient-based outcomes, mainly because of the
increasing need for clinical trials incorporating
QoL measures to be conducted on a multi-national
basis, especially in Europe (Kuyken et al., 1994;
Orley and Kuyken, 1994; Shumaker and Berzon,
1995). As a result, there are quite elaborate
guidelines available intended to ensure high standards
of translation of questionnaires (Bullinger
et al., 1995; Leplege and Verdier, 1995). Amongst
procedures to improve translation, according to
such guidelines, are: use of several independent
translations that are compared; back-translation;
testing of the acceptability of translations to
respondents. Less attention has been paid to
cultural and linguistic variations within national
boundaries, but it would seem that similar
principles could be applied to increase cultural
applicability. Presently, few patient-based outcome
measures have been translated into the languages
of ethnic minorities in the UK.
An important issue is whether rigorous translation
can by itself establish the appropriateness of an
instrument to a new cultural context from the
one in which it was developed. Such methods may
not establish whether subjective experiences differ
in terms of salience from one culture to another,
or indeed may fail to identify concerns and experiences
not anticipated in the culture in which an