Russel-Research-Method-in-Anthropology

Structured Interviewing I: Questionnaires 279
to predict the winner of the presidential election. They got back 2.3 million
ballots and predicted Alf Landon over Franklin Delano Roosevelt in a landslide.
Roosevelt got 61% of the vote.
Now, you’d think that 2.3 million ballots would be enough for anyone, but
two things caused the Digest debacle. First, they selected their sample from
automobile registries and telephone books. In 1936, this favored richer people
who tend to be Republican. Second, the 2.3 million ballots were only 23% of
the 10 million sent out. The low response rate biased the results in favor of
the Republican challenger since those who didn’t respond tended to be poorer
and less inclined to participate in surveys (Squire 1988).
How to Adjust for Nonresponse
Skip to 1991. The American Anthropological Association sent questionnaires
to a sample of 1,229 members. The sample was stratified into several
cohorts who had received their Ph.D. degrees beginning in 1971–1972 and
ending in 1989–1990. The 1989–1990 cohort comprised 306 then-recent
Ph.D.s. The idea was to find out what kinds of jobs those anthropologists had.
The AAA got back 840 completed questionnaires, or 68% of the 1,229, and
the results of the survey were reported in the Anthropology Newsletter in May
1991. The response rate is not high for this kind of survey, where respondents
are being asked for information from their own professional organization.
(The U.S. Office of Management and Budget demands a minimum 75%
response rate from survey contract researchers [Fowler 1984:48] and, as we
saw earlier, this is not an excessive demand.)
Now, 41% of those responding from the 1989–1990 cohort said they had
academic jobs. The Anthropology Newsletter didn’t report the response rate
by cohort, but suppose that 68% of the 1989–1990 cohort—the same percentage
as applies to the overall survey—sent back their questionnaires. That’s
208 out of 306 responses. The 41% who said they had academic jobs would
be 85 of the 208 respondents; the other 123 had nonacademic jobs.
Suppose that everyone who didn’t respond (32%, or 98 out of 306) got nonacademic
jobs. (Maybe that’s why they didn’t bother to respond.) In that case,
98 123 221 out of the 306 people in the cohort, or 72%, got nonacademic
jobs that year—not the 59% (100% 41%) as reported in the survey.
It’s unlikely that all the nonresponders were in nonacademic jobs. To handle
the problem of nonresponse, the AAA might have run down a random grab of
10 of the nonresponders and interviewed them by telephone. Suppose that
seven said they had nonacademic jobs. You’ll recall from chapter 7 on sampling
theory that the formula for determining the 95% confidence limits of a
point estimator is: