Australian Politics and Policy - Senior, 2019a

Australian Politics and Policy
The near-universal acceptance and use of representative, random samples is
due to a high profile polling error more than 80 years ago. During the 1936 US
presidential election, the then very popular magazine Literary Digest ran a mailin
survey that attracted more than two million responses. This is a truly massive
sample size (generally a good thing), even by modern standards. Despite this, the
magazine incorrectly predicted a landslide victory for Republican candidate Alf
Landon over incumbent Democratic President Franklin Roosevelt, who decisively
won the election. The reason for the error? The magazine’s very biased sample of
voters. Subscribers to the Literary Digest were predominantly car and telephone
owners – an affluent group of voters who were not representative of the wider
electorate – and Roosevelt’s supporters were under-represented. 36
The attribute that made the Literary Digest samplesolarge–thehugelistof
subscribers who mailed in survey responses – also made it more error prone. It
used a biased sample. The Literary Digest survey is what we call an opt-in survey.
Thisistheothermainformofsampling.
The problem with this form of survey is that often the respondents who choose
to opt-in are different from the population you are trying to study in important
ways that correlate with the outcome you are researching, biasing the results.
Smaller surveys conducted by George Gallup, Archibald Crossley and Elmo Roper,
with samples comprised of randomly selected voters, more accurately predicted
the 1936 election results. 37 Accordingly, opt-in convenience surveys were largely
discarded by researchers in favour of random sampling.
In addition to the nature of the sample, there are also different methodologies
with which to collect a survey sample. The most common forms of surveys are:
• In-person survey: these allow the interviewer to build a personal rapport with
respondentsandgainmorecompleteanswers.Thismethodcanalsoallow
for longer and more detailed surveys, and interviewers can use visual aids.
However, in-person surveys are much more expensive than other methods
and are geographically constrained by the area an interviewer can cover. They
also have significant problems with social desirability bias – the tendency of
respondents to answer questions in a way they believe will be viewed favorably
by others, under-reporting potentially undesirable behaviour (e.g. eating junk
food, smoking) and over-reporting what might be construed as good behavior
(e.g. exercising daily, eating well, working hard). Due to the cost involved, this
methodology is not used regularly for surveys in the modern era.
• Mail survey: these surveys have the benefit of being affordable, suffer less from
social desirability bias, as there is no human interviewer directly involved, and
canbelongerthanphonepolls.Asaresult,theyhaveremainedpopularfor
academic surveys. However, as there is no person involved – either on the other
36 Squire 1988.
37 Gosnell 1937.
482