Question and Questionnaire Design - Stanford University

266 Jon A. Krosnick and Stanley PresserIt is useful to think of optimizing and strong satisficing as the two ends of acontinuum indicating the degrees of thoroughness with which the four response stepsare performed. The optimizing end of the continuum involves complete and effortfulexecution of all four steps. The strong satisficing end involves little effort in theinterpretation and answer reporting steps and no retrieval or integration at all.In between are intermediate levels.The likelihood of satisficing is thought to be determined by three major factors:task difficulty, respondent ability, and respondent motivation (Krosnick, 1991). Taskdifficulty is a function of both question-specific attributes (e.g., the difficulty ofinterpreting a question and of retrieving and manipulating the requested information)and attributes of the questionnaire’s administration (e.g., the pace at which aninterviewer reads the questions and the presence of distracting events). Ability isshaped by the extent to which respondents are adept at performing complex mentaloperations, practiced at thinking about the topic of a particular question, andequipped with preformulated judgments on the issue in question. Motivation isinfluenced by need for cognition (Cacioppo, Petty, Feinstein, & Jarvis, 1996), thedegree to which the topic of a question is personally important, beliefs about whetherthe survey will have useful consequences, respondent fatigue, and aspects ofquestionnaire administration (such as interviewer behavior) that either encourageoptimizing or suggest that careful reporting is not necessary.Efforts to minimize task difficulty and maximize respondent motivation are likelyto pay off by minimizing satisficing and maximizing the accuracy of self-reports.As we shall see, the notion of satisficing is useful for understanding why somequestionnaire design decisions can improve the quality of answers.9.2. Open versus Closed QuestionsOne of the first decisions a researcher must make when designing a survey question iswhether to make it open (permitting respondents to answer in their own words) orclosed (requiring respondents to select an answer from a set of choices). Although thevast majority of survey questions are closed, some open questions play prominentroles in survey research, such as those about the most important problem facing thecountry.In order to analyze the answers to open questions, they must be grouped into arelatively small number of categories. This requires the development of a coding scheme;its application by more than one person; and the attainment of a high level of agreementbetween coders. The costs of these procedures, coupled with both the difficultiesinterviewers confront in recording open answers and the longer interview time taken byopen questions, are responsible for the widespread use of closed questions.These practical disadvantages of open questions, however, do not apply tothe measurement of quantities. The answer categories to open questions aboutamounts — for instance, number of doctor visits, hours devoted to housework,dollars spent for a good — are implicit in the question, so no coding is required, and