Question and Questionnaire Design - Stanford University

288 Jon A. Krosnick and Stanley Presser9.8. Recall ErrorAside from motivated misreporting due to concern about social desirability,questions about the past are subject to two major sources of error. The first, andmost fundamental, is comprehension. The query ‘‘During the last month, have youpurchased any household furniture?’’ for instance, may be compromised by varyinginterpretations of ‘‘household furniture’’: some people will think lamps count,whereas others will not, and the same will be true for other purchases, such as beds.Thus, it is critical for questionnaire designers to attend to the ways in which terms areinterpreted by respondents (for which pretesting, the subject of our penultimatesection, can be invaluable). But this is true for all items, not just those about the past.The second major source of error — frailties of memory — is usually of greaterconcern for retrospective items than for other kinds of items. In this section, wereview the questionnaire strategies that have been proposed to minimize recall error. 6At the outset, it is important to note that recall aids can only be effective for materialthat was encoded in memory. Although this point seems obvious, surveys sometimesask for information respondents never knew. Lee et al. (1999), for example, showed thatthe very low accuracy of parents’ reports about their children’s immunizations arosebecause many parents never encoded the information. In these cases, askingrespondents to consult records is probably the only way to improve reporting.When information is encoded in memory, its retrieval is strongly affected by boththe information’s salience and the elapsed time since the information was encoded.Unless the information is both recent and salient, it may not come quickly to mind.Thus, time spent recalling is often related to accurate reporting. Since respondentsmay model their behavior on that of the interviewer, instructing interviewers toread more slowly is one way to promote respondent deliberation (and it hasthe added benefit of making question comprehension easier for respondents). 7Adding redundant phrases or sentences, thereby lengthening the question, maylikewise encourage respondent deliberation by increasing the time available for recall(Laurent, 1972).Other ways to increase the time taken to answer questions include providingexplicit instructions to respondents about the importance of carefully searching their6. The strategies we review generally apply to questions about objective phenomena (typically behavior).For a review of problems associated with the special case of recalling attitudes, see Smith (1984) andMarkus (1986).7. As a reminder to the interviewer of the importance of a slower pace, pause notations may be included inthe text of the question, e.g.: ‘‘In a moment, I’m going to ask you whether you voted on Tuesday,November 5th (PAUSE) which was ____ days ago. (PAUSE) Before you answer, think of a number ofdifferent things that will likely come to mind if you actually did vote this past election day; (PAUSE) thingslike whether you walked, drove or were driven. (PAUSE) After thinking about it, you may realize that youdid not vote in this particular election. (PAUSE) Now that you have thought about it, which of thesestatements best describes you: I did not vote in the November 5th election; (PAUSE) I thought aboutvoting but didn’t; (PAUSE) I usually vote but didn’t this time; (PAUSE) I am sure I voted in theNovember 5th election.’’