Analytic Culture in the U.S. Intelligence Community (PDF) - CIA

CHAPTER FIVE
to human biases to explain unreliable expert predictions. 21 There is general
agreement that two types of bias exist:
• Pattern bias: looking for evidence that confirms rather than rejects a
hypothesis and/or filling in—perhaps inadvertently—missing data with
data from previous experiences;
• Heuristic bias: using inappropriate guidelines or rules to make predictions.
Paradoxically, the very method by which one becomes an expert explains
why experts are much better than novices at describing, explaining, performing
tasks, and solving problems within their domains but, with few exceptions,
are worse at forecasting than are Bayesian probabilities based on
historical, statistical models. A given domain has specific heuristics for performing
tasks and solving problems, and these rules are a large part of what
makes up expertise. In addition, experts need to acquire and store tens of thousands
of cases in order to recognize patterns, generate and test hypotheses, and
contribute to the collective knowledge within their fields. In other words,
becoming an expert requires a significant number of years of viewing the
world through the lens of one specific domain. This concentration gives the
expert the power to recognize patterns, perform tasks, and solve problems, but
it also focuses the expert’s attention on one domain to the exclusion of others.
It should come as little surprise, then, that an expert would have difficulty
identifying and weighing variables in an interdisciplinary task, such as forecasting
an adversary’s intentions. Put differently, an expert may know his specific
domain, such as economics or leadership analysis, quite thoroughly, but
that may still not permit him to divine an adversary’s intention, which the
adversary may not himself know.
The Burden on Intelligence Analysts
Intelligence analysis is an amalgam of a number of highly specialized
domains. Within each, experts are tasked with assembling, analyzing, assigning
meaning to, and reporting on data, the goals being to describe an event or
observation, solve a problem, or make a forecast. Experts who encounter a
case outside their field repeat the steps they initially used to acquire their
expertise. Thus, they can try to make the new data fit a pattern previously
acquired; recognize that the case falls outside their expertise and turn to their
domain’s heuristics to try to give meaning to the data; acknowledge that the
21
J. Evans, Bias in Human Reasoning; R. Heuer, Psychology of Intelligence Analysis; D. Kahneman,
P. Slovic, and A. Tversky, Judgment Under Uncertainty; A. Tversky and D. Kahneman,
“The Belief in the ‘Law of Small Numbers’.”
66