Asking Questions - The Definitive Guide To Questionnaire Design ...

112 ASKING QUESTIONS
Determining the Perceived Threat of Questions
It is often very useful to determine at the end of an interview which
questions were considered threatening or hard to understand. An
example of a series of questions we have used for this purpose is
given in Figure 3.10. The most useful of these is Question 4, which
asks respondents to indicate whether they thought the questions
“would make most people very uneasy, moderately uneasy, slightly
uneasy, or not at all uneasy.” Note that this is a projective question
about most people and is less threatening than the direct questions
asking respondents to report about their own uneasiness.
Such questions can be used not only to determine general levels
of threat but also as an indicator of respondent veracity. Respondents
who report that the question would make most people uneasy
are more likely to underreport than are other respondents.
Use Additional Sources to Validate Accuracy
Although validation from outside sources is always valuable in surveys
of behavior, it is particularly important to validate the level of
threat associated with a given behavior. As we have seen in this
chapter, overreporting and underreporting can be dealt with in various
ways, but there is still not enough research to predict in specific
cases how big an effect these procedures will have. Moreover, some
behaviors such as sexual activity, by their very nature, are private,
and no outside validation is possible. Where it is possible, however,
validation provides a procedure for evaluating results obtained from
alternative methods and ultimately leads to better questionnaires.
Validation at an individual level is most powerful but also most
difficult. It is possible, for example, to compare individual reports of
doctor and hospital visits to records of medical care providers or
insurers, but that requires permission from the individual and cooperation
from the provider. It must be remembered that record information
is also incomplete and variable in quality. Such methods as
chemical analysis of hair, saliva, and urine samples have been devel-