Asking Questions - The Definitive Guide To Questionnaire Design ...

168 ASKING QUESTIONS
they think are the most desirable. From these three, they are asked to
pick the one that would be most desirable, thus establishing rank one.
If it is important to establish second- and third-place rankings, respondents
may pick among the remaining two what they think is the
next desirable. The third-ranked quality would thereby be established
by elimination. Respondents are also asked to pick the three least
desirable qualities and, among those three, the one they consider the
least important. In this manner fairly clear differentiations can be
built up at each end of the scale, but no differentiation in the middle.
Sometimes respondents find ranking difficult to do and will
select no more than a first choice. Rankings are particularly difficult
for respondents when the items to be ranked are quite different
from one another (such as policy preferences) or are all either very
desirable or very undesirable.
Ranking Through Paired Comparison
Another method for obtaining rankings, which is not often used in
surveys but which we think could be more widely used, is paired
comparisons. Each pair of alternatives is compared and ranked
according to preference. Several examples involving paired comparison
are given in Figure 5.5. Questions 5 through 7, for example,
involve three response alternatives. The paired-comparison method
has the advantage that the respondent considers each alternative
in comparison with each other alternative, one at a time. Respondents
can consider preferences in a more discrete fashion. It is a particularly
good method when the choices are among objects that are
all desirable or undesirable.
If there is a consistent preference structure, the method should
obtain a transitive ordering of the alternatives; that is, if A is preferred
to B and B is preferred to C, then A should be preferred to C.
Sometimes, however, it turns out that the ordering is not transitive:
A is preferred to B, B is preferred to C, but C is preferred to A. The
straight ranking method forces transitivity on the rankings. If you
use that method, you might fail to discover that people’s preferences