i.F ..nll..i - US Watch

Page 3
Amencas and South Paufic
Home Nolh Amei'cii ' Urilr-:l Sialcg
US election cycle highlights key analytical pitfalls
Obtaining strong qualitative anaiysis of politics in developed countries. particularly the
United States, should be relatively easy. Interest in politics and election outcomes is
high, the market for good information is enormous and lucrative, and the political
System is awash in money. In addition, the Unibd States has both a saturated media
industry and the largest professional polling market in the world Yet last November, a
substantial segment of the public and professional political observers were genuineiy
surprised by the election ou!come --though it was not remoteiy surprising. Examining
how. and why, political analysts have made poor predictions even in very favourable
conditions can help highlight relatively simpie means of improving analyttcal outcomes.
and of avoiding common mistakes.
What next
Election prediction in most heavily polled federai races is not particularly difficuit - if
several key qualitative analytical principles are followed. Even on poiicy questions
where polling metrics are less relevant and outcomes more uncertain, maintaining
discipline in assimilating new information, ignoring dublous narratives, and attempting to
account for uncertainty can usually provide strong insight into likely outcomes.
Analysis
The Republican establishment has spent the past five inonths excoriating itself for not
identifying, relatively early on: that the 2012 election cycle was unlikely to be favourable
for the party. Yet Republican leaders do not have a monopoly on bad analysis of US
politics. Despite devoting huge sums to political analysis, much of the output de!ivered
to both parties and the public is poor. By keeping a few key principles in mind. it is
possible to draw useful lessons on how to improve political trend analysis -- and
qualitative analysis more broadly.
Causes of analytical failure
Impact
. Political poiarisation has
generally made identifying
US political and election
trends easier, not harder.
. An early inference that the
2014 midlerm election cycle
is iikeiy to be a relatively
strong one for Republicans
is supportable.
. However, absent a
significant change in
strategy and meaningful
platform rev!sions, long-term
trends look unfavourable for
Repubilcans
Broadly speaking, there are three primary contributors to bad US political analysis
Ideological factors
The most obvious factor is ideology - but not in the sense that political analysts
ConscioiIsIy distort their views to favour one party's interest. (While this sometimes
occurs, knowledgeable, skiiled analysts -such as George Wil and Michael Barone --
would not have risked severe professional embarrassment by deliberately skewing their
pre-election assessments.) Rather. many analysts are unable to separate their view of
the world from polling and political trends that are not always congruent with it^
This is a variation on the problem created by strong 'analyticai frameworks' (see
INTERNATIONAL: Research shows why strategic plans fail -May 4, 2012) analysts
can become accustomed to relying on a framework for viewing a particular problem and
unconscioitsly discount new information that does not 'fit the framework'.
'Horse race' narratives
O Oxford Analytic8 2013. All rights reserved
NO duplication or transmission of this doci!me!it is permitted without the written consent of Oxford Analyllca
C ~ ~ . i.F l ~ ..nll..i ~ l rnml.l\nlilir~nt.ir+r roll -Ad ISEF. If < Gnn nr ;& hlnith 7 Uiln OEF; 71iP

Page 4
Amedcxsand SoUth PaMc
The US media oflen contributes to these false narratives because its prtmaly objective
(telling an exciting political story) differs from that of professional analysts (gauging the
likely outcome of the election). Thus, in relatively static elections such as the 2012
presidential contest: the media has a strong iiicentive to over-interpret small, frequently
insignificant, polling movements as indicative of a major shifl in trend.
Misuse of history
Finally, it is tempting to misuse historical 'precedent' In analysis of US elect~ons In the
history of modern polling there have been just 20 presidential election cycles. Seeking
to draw conclusions utilising a dataset with so few data points should be undertaken
with considerable caution.
However, political analysts often 'cherry pick' this iimited historical data to support
sometimes dubious conclusions. For exampie. during the 2012 cycle some analysts
emphasised the fact that no tncumbent president had won re-election since 1940 when
the official unemployment rate exceeded 7.3%. Yet given the iimited dataset and the
fact that Franklin Roosevelt won re-election (1836 and 1940) twice when unemployment
was at least double that rate (16.6% and 14.6% respectively). the usefulness of this
historical correlation was certainly over-emphasised (see UNITED STATES: Romney
faces uphill campaign fight - August 27. 2012) and see UNITED STATES: Ohama
retains edge over challengers - December 16, 2011).
Improving analysis
VVith a little effort to contain these negative tendencies. the accuracy of politicai trend
analysis can be improved.
Information assimilation
As statistician and polling analyst Nate Silver explained in his 2012 book. 'The Signal
and the Noise', good qualitative anaiysts adjust their assessments in response to
additional data. In contrast, poor analysts have a tendency to stlck to their previous
conclusions regardless of new data -- or, even worse: interpret all new data in a manner
that simply reinforces their previous conclusions (behaviourai economists call this
'confirmation bias').
One of the best qualitative anaiysts of US politics. Charlie Cook of the Cook Political
Report, has since 1984 been correct approximately 92.95% of the time in competitive
congressional and presidential races. In early 2011, he suggested that 2012 couid be a
good year for Repubiicans. Yet, critically. as more polling atid other data becanie
available, his assessment shifted completely -- and ended up being very accurate
several weeks before 2012 eleclions
Weighing data
Not all data is equally significant. While US pollsters hardly have a perfect record. they
are both the single most reliable barometer of publlc opinion and an accurate guide to
nationai election outcomes - when their resuits are assessed a few days ahead of an
election and aggregated. (On the other hand. cherry pick~ng individual polls -- some of
which will necessarily be highiy inaccurate outliers - freqiiently leads to serious
failures.) There have been a few cases when polling has, for various reasons, been
systematically wrong: but if an analyst believes this is happen~ng it is incumbent ilpon
them to explain why.
@Oxford Analvlica 2013. All rights reserved

O Oxford Analytica 2013. All rights reserved
No duplication or tra!rsmission of this document is perlnitted without the written consent oiOxford Analytica
Cnntnd iua. www oran cornlaholrtlcontartn or rnil +44 lilt35 261 ROO oi in Norllr America 1 800 965 766
Page 5
Arnerlcasand South Pauflc
In the current, highly polarised, political environment, demographic data is also a useful
analytical guide. since the behaviour of particular voting blocs is increasingly consistent.
However, the media teiids to downplay polling and demographic data in favour of a focus
on campaign strategy and tactics. In local campaigns, these strategies can be highlight
significant to the outcome, but in presidential contests there is relatively little polling
data to support the notion that campaign tactics had a decisive impact on the general
election outcome since the late 1980s.
Accounting for uncertainty
Finally. in is incumbent upon an analyst to gauge, and be clear about, the uncertainty
inherent in any political trend assessment. Such uncertainty is, of course. much greater
the further into the future the projection: more than two months before the polls, any
projection should be treated as highly speculative. However, as Election Day
approaches, the risk of disruptive contingencies subsides and polling data has far more
predictive power. At that point, if the aggregate polling data indicates a clear leader. it
becomes disingenuous and potentially unsupportable to maintain that the race is 'too
close to call'.
Outlook
Media fragmentatlon, a changing business model for pollsters, and an increasingly
poiarised and ideologically entrenched electorate means that the overall quality of US
political analysis is likely to remain poor. Nevenheless there are a small number of
exceptionally high quality analysts at work in this discipline; all tend to alter the~r
conclusions when confronted with new data, assimilate information effectively, and have
a solid grasp of how to measure uncertainty
Word Counf (approx): 1203