Asking Questions - The Definitive Guide To Questionnaire Design ...

ASKING NONTHREATENING QUESTIONS ABOUT BEHAVIOR 73
following examples are taken from Payne’s Rogue’s Gallery of Problem
Words (1951, chap. 10):
Any, anybody, anyone, anything. May mean “every,” “some,”
or “only one.”
Fair. Meanings include “average, pretty good, not bad,”
“favorable, just, honest,” “according to the rules,” “plain,”
“open.”
Just. May mean “precisely,” “closely,” “barely.”
Most. A problem if it precedes another adjective, as it is not
clear whether it modifies the adjective or the noun, as in
“most useful work.”
Saw, see, seen. May mean “observe” or may mean “visit a
doctor or lawyer.”
Other words may have unexpected meanings to some respondents.
A careful pilot test conducted by sensitive interviewers is the
most direct way to discover these problem words. Since respondents’
answers may not always reveal their possible confusion about
meanings, it is often useful to ask a respondent at the end of a pilot
test “What did you think we meant when we asked [word or
phrase]?”
Determine the Appropriate Length of Questions
It has generally been the practice to make questions as short as possible.
This practice was based on research on attitude questions,
which indicated that response reliability declines as the length of
the question increases. Research on behavior questions, however,
indicates that the findings for attitude questions do not apply to
behavior questions (Cannell, Marquis, and Laurent, 1977; Cannell,
Oksenberg, and Converse, 1977; Bradburn, Sudman, and Associates,
1979). For behavior topics, longer questions help reduce the
number of omitted events and thus improve recall. There are three
main reasons why longer questions improve recall.