Asking Questions - The Definitive Guide To Questionnaire Design ...

234 ASKING QUESTIONS
for instance, usually produces a colorful array of responses in the student’s
“own words,” but provides very little information that can be
used for formative evaluation. Instructors still value these responses
because they can attach their own interpretation to these comments.
Using closed-ended (limited-response) formats provides
more accurate counts of the types of responses to each item. In
many situations, the best approach is to use a combination of both
closed-ended and open-ended responses.
Most student evaluation forms suffer from severe design problems.
This is typically because they are designed by committees of
students or administrators who have little questionnaire design
experience and little regard for the unforeseen consequences of a
poorly worded or inaccurate question.
The type of question being asked will typically determine the
type of closed-ended responses the survey designer can provide. As
with other surveys, if appropriate responses do not match each question,
incongruous and unreliable responses will result. For instance,
if a continuum-response format is used and only endpoints are
anchored (for example, Very Poor 1–2–3–4–5–6–7 Very Good),
it will tend to produce unreliable responses because it is not clear
what the midpoint represents. It is important that each response
point along the continuum be appropriate whenever an item is
stated either positively or negatively. Another type of possible
response scale requires elaborate behavioral descriptions along the
continuum. (See the discussion of BARS earlier in this chapter.) If
possible, items with common responses should be grouped together.
Selecting the Appropriate Items
One way to generate a questionnaire draft is to evaluate what other
institutions are using and to adopt similar ideas. Many institutions
have catalogues of questions (often called items) that have been
tested previously. Care must be taken before adopting another set
of survey items unless it is known whether the other institution
considered the possible shortcomings discussed earlier and evalu-