Thinking and Deciding

METHODS FOR EMPIRICAL RESEARCH 49
time they get to the second. This is the approach used by Baron and Hershey (1988)
in most of their experiments. In this kind of design, it is necessary to counterbalance
or randomize the order of the S and F forms of each case. In counterbalancing, half
the subjects get the S form first and half get the F form first. In randomization, the
order of the two forms of each case is determined at random for each case, ideally for
each subject. These procedures are necessary to make sure that any results are not
caused by order of presentation. Subject may, for example, rate earlier cases higher
(or lower) than later cases.
Sampling
Researchers from different traditions have very different approaches to the sampling
of subjects. Sociologists and public-opinion researchers go to great lengths to draw
random samples from some population, such as the adult citizens of some particular
nation. Psychologists often use what others call (somewhat derisively) “convenience
samples,” such as students enrolled in an introductory psychology course.
When you want to say something about a particular population, you need to
sample that population in a representative way. You must choose respondents at
random and try to get them all to respond. If they do not respond, you must try to
show that the ones who did not respond would not have answered differently from
the ones who responded. Much research has been done on how this should be done
(Dillman, 1978).
Little of the research described in this book is concerned with any particular population.
It is concerned with people in general. The studies are designed to demonstrate
some effect, such as the outcome bias just described. Then further studies are
done to analyze the effect. All that is required for these studies is some group of
subjects who show the effect in question. Most researchers are not very interested in
the prevalence of the effect. (But see the earlier discussion of individual differences.)
Suppose the population of interest is human beings. If we are reasonably optimistic
about the future, most human beings have not yet been born. It is thus
impossible to sample them randomly. Although this point is sort of a joke, there is
a serious part to it. Efforts to sample some particular nation are misdirected if we
are really interested in a broader population. If we want to make sure that our results
are not due to some idiosyncrasy of our sample, a general way to do this is to examine
very different samples, particular people from different cultures. Often cultural
comparisons can be enlightening in other ways.
The recent growth of the Internet has provided new opportunities for recruiting
subjects. As more and more people around the world get access to the World Wide
Web, for example, it has become possible not only to recruit convenience samples
on the Internet but also to recruit specialized samples, such as cancer patients or
citizens of India. Even the convenience samples tend to be more varied than the
college students typically used in research. It is possible to test for effects of age, for
example, which varies very little among students.