Evaluating Patient-Based Outcome Measures - NIHR Health ...

40
Criteria for selecting a patient-based outcome measure
examined than issues such as reliability and
validity, and there is less consensus as to what
constitutes acceptability. For Selby and Robertson
(1987), acceptability is ‘a description of the speed
of completion of the questionnaire and the proportion
of patients who find it difficult, impossible or
unacceptable for any reason’ (1987:528). Ware and
colleagues (1981: 622) subsume these properties
under practicality: ‘An important aspect of practicality
is respondent burden, indicators of which
include refusal rates, rates of missing responses,
and administration time’. Pragmatically, trialists
using patient-based outcome measures are concerned
with the end result; whether they obtain
as complete data from patients as possible.
Methods for increasing completion rates have
been addressed in a number of reviews (Yancik
and Yate, 1986; Aaronson, 1991; Sadura et al.,
1992). Others have considered how to analyse
missing data (Fayers and Jones, 1983; Zwinderman,
1990). However, we need to consider the
different components of acceptability in turn
to identify sources of missing data.
Reasons for non-completion
Patients may either not return a whole assessment
or may omit some items in the assessment. If
patients either do not attempt to complete an
instrument at all or omit particular items frequently,
this is potentially a sign that a questionnaire
is difficult to understand, distressing, or in some
other way unacceptable. It may also be evidence of
poor validity of an instrument if the non-response
rate is high. However, there may be other reasons
for non-completion such as the method of delivery
of the questionnaire. Patients may not receive a
mailed questionnaire in the first place or may not
have a telephone in order to be contacted in this
way. Patients may also be unable to complete
questionnaires because of their health status or
other disabilities, particularly cognitive or visual
(Medical Research Council, 1995). In practice,
determining the role that an instrument has on
completion rates compared to other factors is
not easy.
It is beyond the scope of this report to consider
broader issues of survey methodology. However,
it is important to be aware of the evidence that
how a questionnaire is administered can influence
response rates regardless of content. Postal surveys
are more often used because they are cheaper
than alternatives. However, they tend to have lower
response rates than personally administered or
telephone interviews. It has been argued that, if
careful attention is paid to methodology, general
postal surveys can expect to achieve 75–80%
response rates and a variety of extra steps may
be used to increase this level (Dillman, 1978;
de Vaus, 1986; Oppenheim, 1992). Surveys using
patient-based outcomes are subject to the same
effects with postal methods of data collection
achieving somewhat lower response rates than
other methods (Sullivan et al., 1995; Weinberger
et al., 1996).
It may be noted that there is also an unresolved
debate in the general survey literature as to
whether the method of administration can influence
the content of answers to a questionnaire
(Bremer and McCauley, 1986; Anderson et al.,
1986; Chambers et al., 1987). In the general
survey literature, there is substantial evidence
that respondents give more favourable reports
about aspects of their well-being when personally
interviewed than they provide in a self completed
questionnaire (Schwarz and Strack, 1991). Topics
of a particularly sensitive nature are considered
particularly prone to effects of method of data
gathering, but the evidence is inconsistent as to
whether mailed questionnaire or interview
produce more accurate information (Wiklund
et al., 1990; Korner Bitensky et al., 1994). Cook
and colleagues (1993) showed the significance
of this factor in patient-based outcome measures;
patients reported more health-related QoL
problems on a self completed questionnaire
than when personally interviewed.
More general features of the layout, appearance
and legibility of a questionnaire are thought to
have a strong influence on acceptability. Some
instruments such as the COOP Charts have deliberately
included extremely simple and short forms
of wording of questions together with pictorial
representations to add to ease and acceptability
of use (Hughes et al., 1995). A rare experimental
study to test the benefit of pictorial representation
in a QoL study showed that cartoon figures to
depict degrees of illness severity improved test–
retest reliability compared with responses to
conventional formatting (Hadorn et al., 1992).
The health status of respondents can influence
the likelihood of completing a questionnaire.
Hopwood and colleagues (1994) provide evidence
that, in a sample of patients with lung cancer,
completion rates of a health status questionnaire
were 92% amongst patients independently assessed
as in the most favourable health state but 31%
amongst those in the poorest health state. Poorer
visual function has also been shown to be an
influence on non-response in health status surveys
(Sullivan et al., 1995). There is conflicting