Asking the Right Questions, A Guide to Critical Thinking, 8th Ed

124 Chapter 9
generalize have an equal chance of getting sampled; they try to avoid a
biased sample. Major polls, like the Gallop poll, for example, always try
to sample randomly. This keeps them from getting groups of events
or people that have biased characteristics. Do you see how each of the
following samples has biased characteristics?
a. People who volunteer to be interviewed about frequency of sexual
activity.
b. People who are at home at 2:30 P.M. to answer their phone.
c. Readers of a popular women's magazine who clip and complete
mail-in surveys.
Thus, we want to ask of all research studies, "How many events or people
did they sample, how much breadth did the sample have, and how random
was the sample?"
A common problem that stems from not paying enough attention to the
limits of sampling is for communicators to overgeneralize research findings.
They state a generalization that is much broader than that warranted by the
research. In Chapter 7, we referred to such overgeneralizing as the Hasty
Generalization fallacy. Let's take a close look at a research overgeneralization:
Alcohol consumption is at an all-time high at colleges nationwide. A recent survey
conducted by Drinksville University found that of the 250 people surveyed,
89 percent drink on a semi-regular basis.
Sampling procedures prohibit such a broad generalization. The research
report implies the conclusion can be applied to all campuses, when the research
studied only one campus. We don't even know whether the conclusion can be
applied to that campus, because we don't know how randomly researchers
sampled from it. The research report is flawed because it gready overgeneralizes.
Remember: We can generalize only to people and events that are like those
that we have studied in the research!
Biased Surveys and Questionnaires
It's early evening. You have just finished dinner. The phone rings. "We're conducting
a survey of public opinion. Will you answer a few questions?" If you
answer "yes," you will be among thousands who annually take part in surveys—
one of the research methods you will encounter most frequently. Think how
often you hear the phrase "according to recent polls."