Asking Questions - The Definitive Guide To Questionnaire Design ...

ASKING QUESTIONS THAT MEASURE KNOWLEDGE 205
tions about new products. Some advocates of particular public
policies have attempted to discredit public opinions that oppose
their policies by demonstrating that public knowledge of the
issues is limited. Although this can sometimes be legitimate, the
difficulty level of the questions must also be taken into account.
It is always possible to find questions so difficult that virtually no
respondents can answer them correctly—especially in a survey
where an instant response is required and no advance warning has
been given.
Reducing the Threat of Knowledge Questions
As with the threatening behavior questions discussed in the previous
chapter, knowledge questions raise issues of self-presentation.
Respondents do not wish to appear foolish or ill-informed by giving
obviously incorrect answers or admitting to not knowing something
that everyone else knows. Much of this threat can be reduced by an
introductory phrase such as “Do you happen to know” or “Can you
recall, offhand.” Offering “I don’t know” as an answer category also
reduces threat. These procedures indicate that a “don’t know”
answer is acceptable even if it is not the most desirable answer. The
use of these threat-reducing phrases reduces the amount of guessing
and increases the percentage of “don’t know” answers. Conversely,
if you wish respondents to give their “best guess,” the phrases used
above should be omitted, and respondents should be asked to give
“your best guess,” as in this Gallup question: “Just your best guess,
what proportion of persons on welfare are ‘chiselers,’ that is, are collecting
more than they are entitled to?”
The line between knowledge and attitude or opinion questions
is often blurred. Earlier (in Figure 6.11, Questions 1 and 14),
knowledge questions are asked in the guise of opinion questions.
The question that asks respondents to guess about the proportion
of welfare chiselers is really an attitude question in the guise of
a knowledge question. Although a few respondents may actually
know the correct proportion from reading news stories, most