Asking Questions - The Definitive Guide To Questionnaire Design ...

ASKING QUESTIONS ABOUT ATTITUDES AND BEHAVIORS 145
to notice that they have added an extra consideration somewhere
along the line and that, as a result, two attitude objects have become
joined in one question. Whenever—as in the SALT II question—
the reasons for holding opinions appear as qualifications in the question
itself or in a response category, a red flag should go up.
Bias Related to the Context and Meaning
of Adjacent Questions
Interviews are forms of conversations. Whenever they take place
in person or on the telephone, they flow as conversations (albeit
sometimes rather peculiar conversations that are dominated by the
question-and-answer sequence). Survey participants are responding
not only to the actual questions but also to the context in which the
questions are being asked. This includes such things as the stated
purpose of the interview, the sponsor of the survey, the topics being
asked, the norms of ordinary conversation between strangers, and
other factors. Because questions are asked sequentially, answers to
questions trigger thoughts in respondents’ minds that may spill over
and influence the answers to later questions.
The potential biasing effect of the positioning of questions in a
questionnaire has long been recognized as a problem in survey and
market research. Since the earliest days of survey research, studies
of question order have produced both positive and negative results.
Although we still do not understand many of the processes
involved in order effects, research on cognitive aspects of surveys
have enabled us to better understand the effect of order and gives
us guidance about where to expect such effects. We can describe situations
that should alert the investigator to the possibility of order
effects. (The order problem is also discussed in Chapter Ten in the
discussion of funneling and reverse funneling. For more detailed
explanation of the cognitive mechanisms that create context and
order effects, see Sudman, Bradburn, and Schwarz, 1996.)
Why should order matter? Stating explicit alternatives provides
a context or framework within which the respondent answers