Asking Questions - The Definitive Guide To Questionnaire Design ...

194 ASKING QUESTIONS
It must be recognized, however, that residents, including community
leaders, are not merely disinterested observers. They have
large emotional stakes in their communities. Answers to factual
questions may be affected by attitudes as well as level of knowledge.
Thus, single responses about a neighborhood may not be correct.
Averaging or otherwise weighting the responses from the same
neighborhood increases both reliability and usefulness.
Knowledge of Occupations
Figure 6.10 presents a series of questions used to determine how
much people know about various jobs. The principal reason for
these questions is to help explain how different people rate the prestige
of different occupations. Obviously, one factor involved in rating
is knowledge. Note that there are five dichotomous (yes-no)
questions for each job. A respondent should be able to get about
half the answers right simply by guessing. Thus, it is the total right
answers to all ten jobs that discriminates between respondents, and
not the right answers to a single question or about a selected job. It
is also possible to compare public familiarity with individual jobs,
although this was not a primary purpose of these questions.
This nature of knowledge question—the aggregate knowledge
question—is often useful when trying to determine one’s general
level of knowledge in an area. It is particularly useful when the basic
knowledge question that would be asked is generally a binary one
(yes or no, Republican or Democrat, and so on) that would lend
itself well to guesswork.
National Assessment of Educational Progress
The most ambitious program to measure the effects of public education
in the United States has been the National Assessment
of Educational Progress, a multimillion-dollar project of the U.S.
Department of Education. Figure 6.11 presents a series of exercises