Analytic Culture in the U.S. Intelligence Community (PDF) - CIA

DEFINITIONS
Error is reporting the wrong thing.
A failure is something catastrophic, and an error is just a mistake.
Error is about facts; failure is about surprise.
Error is when nobody notices, and failure is when everybody
notices.
Some responses disavowed the existence of intelligence error and failure;
some placed the terms in the broader context of policy and decisionmaking;
some interchanged the two terms at random; some defined the terms according
to their outcomes or consequences. Despite the variability of the
responses, two trends emerged: novice analysts tended to worry about being
factually inaccurate; senior analysts, managers, and consumers, tended to
worry about being surprised. Often, participants’ responses were not definitions
at all but statements meant to represent familiar historical examples:
The attack on Pearl Harbor.
The Chinese sending combat troops into Korea.
The Tet Offensive.
The Soviet invasion of Afghanistan.
The collapse of the Soviet Union.
The Indian nuclear test.
September Eleventh.
The danger of defining by example is that each case is contextually unique
and can be argued ad infinitum. What is important about these examples as a
whole is that they all indicate one central and recurring theme. Specifically, all
these examples signify surprise—in some cases, intelligence surprise; in other
cases, military, civil, and political surprise. Even if the Intelligence Community
itself was not surprised by one of these events, it was unable to convince
the military, civil, and political consumers of intelligence that these events
might occur; in which case, the failure was one of communication and persuasion.
When I began this study, my own definition of error and failure derived
from the psychological and cognitive disciplines. Specifically, I took it that
human error and failure are related to measures of cognitive and psychomotor
accuracy, commission of error being at one end of the accuracy scale and
omission or not performing the correct action being at the other. 6
6
See Appendix A for a list of literature on error.
7