Confirmation Bias: A Ubiquitous Phenomenon in Many Guises

196 RAYMOND S. NICKERSON
benefit of the doubt, so long as there is room for
doubt and, in some cases, even when there is
not. The usual strategy for dealing with
anomalous data is first to challenge the data
themselves. If they prove to be reliable, the next
step is to complicate the existing theory just
enough to accommodate the anomalous result
to, as T. S. Kuhn (1970) put it, "devise
numerous articulations and ad hoc modifications
of [the] theory in order to eliminate any
apparent conflict" (p. 78). If that proves to be
too difficult, one may decide simply to live with
the anomaly, at least for a while. When a theory
is confronted with too many anomalies to be
accommodated in this way—or when, as a
consequence of a series of modifications the
theory becomes too convoluted to manage and
an alternative theory becomes available—there
is the basis of a paradigm shift and a
revolutionary reorientation of thinking.
Overconfidence. Overconfidence in experimental
results has manifested itself in the
reporting of a higher-than-warranted degree of
certainty or precision in variable measurements.
Scientific investigators often have underestimated
the uncertainty of their measurements and
thus reported errors of estimate that have not
stood the test of time. Fundamental constants
that have been reported with uncertainty estimates
that later proved too small include the
velocity of light, the gravitational constant, and
the magnetic moment of the proton (Henrion &
Fischhoff, 1986).
The 1919 British expedition to West Africa to
take advantage of a solar eclipse in order to test
Einstein's prediction that the path of light would
be bent by a gravitational field represents an
especially noteworthy case of the reporting of a
higher-than-warranted degree of precision in
measurement. Einstein had made the prediction
in the 1915 paper on the general theory of
relativity. Scientists later discovered that the
error of measurement was as great as the effect
that was being measured so that, as Hawking
(1988) put it, "The British team's measurement
had been sheer luck, or a case of knowing the
result they wanted to get, not an uncommon
occurrence in science" (p. 32).
The predictions have subsequently been
verified with observations not subject to the
same measurement problems, but as first made
and reported, they suggest the operation a
confirmation bias of considerable strength. In a
detailed account of the event, Collins and Pinch
(1993) noted that Eddington's data were noisy,
that he had to decide which photographs to
count and which to ignore, and that he used
Einstein's theory to make these decisions. As
they put it:
Eddington could only claim to have confirmed Einstein
because he used Einstein's derivations in deciding what
his observations really were, while Einstein's derivations
only became accepted because Eddington's
observation seemed to confirm them. Observation and
prediction were linked in a circle of mutual confirmation
rather than being independent of each other as we
would expect according to the conventional idea of an
experimental test. (p. 45)
Collins and Pinch's account of the reporting
of the results of the 1919 expedition and of the
subsequent widespread adoption of relativity as
the new standard paradigm of physics represents
scientific advance as somewhat less inexorably
determined by the cold objective assessment of
theory in the light of observational fact than it is
sometimes assumed to be.
Henrion and Fischhoff (1986) suggested that
the overconfidence associated with the estimates
they considered could have resulted from
scientists overlooking, for one reason or another,
specific sources of uncertainty in their
measurements. This possibility is consistent
with the results of laboratory studies of judgment
showing that people typically find it easier
to think of reasons that support a conclusion
they have drawn than to think of reasons that
contradict it and that people generally have
difficulty in thinking of reasons why their best
guess might be wrong (Koriat et al., 1980).
By way of rounding out this discussion of
confirmation bias in science, it is worth noting
that prevailing attitudes and opinions can
change rapidly within scientific communities, as
they can in other communities. Today's revolutionary
idea is tomorrow's orthodoxy. Ideas
considered daring, if not bizarre or downright
ridiculous when first put forward, can become
accepted doctrine or sometimes obvious truths
that no reasonable person would contest in
relatively short periods of time. According to
Lakatos (1976)
Newton's mechanics and theory of gravitation was put
forward as a daring guess which was ridiculed and
called "occult" by Leibniz and suspected even by
Newton himself. But a few decades later—in absence
of refutations—his axioms came to be taken as
indubitably true. Suspicions were forgotten, critics