Evaluating Patient-Based Outcome Measures - NIHR Health ...

instrument was first developed (Hunt, 1998).
In this sense even the most thorough observation
of translation procedures cannot alone establish
the validity of an instrument in a new culture.
An unusual solution that attempts to overcome
the cultural specificity of questionnaires is the
WHOQOL Group’s (1998) development of the
World Health Organization Quality of Life Assessment
(WHOQOL). Instead of the usual practice
in which a questionnaire is developed in one
culture and then translated into the languages of
other cultures, in this case concepts and questionnaire
items were developed in 15 different field
centres around the world, including developing
as well as developed countries. Initial results have
appeared regarding basic aspects of reliability
and validity (WHOQOL, 1998). Further research
will be required to examine the value of this
100-item questionnaire.
Summary
Evidence is required that an instrument is acceptable
to patients. The simplest and most direct form
of such evidence is that it has consistently been
associated with high response rates. Early on in the
development of an instrument, this property may
have been more directly tested by eliciting views
of patients about the instrument.
Feasibility
Is the instrument easy to administer
and process?
In addition to patient burden and acceptability,
it is important to evaluate the impact of different
patient-based outcome measures upon staff and
researchers in collecting and processing information
(Aaronson, 1992; Lansky et al., 1992;
Erickson et al., 1995). Data from patients for
clinical trials are often gathered in the context of
regular clinical patient care and excessive burden
to staff may jeopardise trial conduct and disrupt
clinical care. An obvious example is the additional
staff effort and costs involved in personally administering
questionnaires over postal delivery. To a lesser
extent, the length and complexity of instrument
are an additional component. Certainly it may
require additional staff time to assist and explain
how more complex questionnaires are to be filled
out by patients. The simplest of instruments such
as the nine-item COOP charts require a minimum
Health Technology Assessment 1998; Vol. 2: No. 14
of time and effort to process (Nelson et al., 1990).
Their brevity (one item per domain) and pictorial
representation mean that they require less staff
supervision than most alternatives. A related
component of feasibility is time required to
train staff to use an instrument, with questionnaires
designed for self completion imposing the least
burden in this respect. Where instruments do
require interviewer administration, training needs
can vary according to the complexity of the tasks.
Read and colleagues (1987) compared the training
times required for three health status instruments
and found that they varied from 1 to 2 hours for
the easiest to 1 to 2 weeks for the most complex
instrument. Utility measures which involve respondents
making complex judgements under unusual
experimental conditions almost invariably require
highly trained staff (Feeny and Torrance, 1989).
It is sometimes thought that more complex scoring
systems reduce feasibility compared to simple
scores. However, with computer programmes
universally used to process such data, this element
is unlikely to be a major component of burden to
staff. Far more likely to require time to process are
the measurement of physical marks put by patients
onto visual analogue scales which have directly to
be measured in terms of distance from origin.
Above all, with both acceptability and feasibility,
as with other dimensions we have examined, these
should not be considered entirely fixed properties
of instruments. To some extent, both the content
and appearance of instruments can be improved
to enhance response rates. Probably more importantly,
as Bernard and colleagues (1995) argued in
their qualitative study of the use of health status
measures, staff attitudes and acceptance of the
value of patient-based outcome measures can
make a substantial difference to ultimate
acceptability by patients.
Summary
The time and resources required to collect, process
and analyse a patient-based outcome measure are
not often independently reported so that evidence
may not be readily available. A judgement of this
aspect of an instrument has to be made in the
context of clinical trials given that this will be but
one component of burden on participants that
will determine the overall viability of a trial and
therefore the quality of its final results.
43