PART I. DISCOVERY — PROOF — CHOICE By Frank T. Hughes (Originally disseminated as class notes for his JMIC class on The Art and Science of the Process of Intelligence Analysis.) Information and expertise are a necessary but not sufficient means of making intelligence analysis the special product that it needs to be. A comparable effort has to be devoted to the science of analysis. 125 —Douglas MacEachin Former DDI, CIA This course concerns the marshaling of intelligence evidence and the construction of defensible and persuasive arguments based on this evidence. Both of these tasks are crucial in effective intelligence analysis and form the very foundation for later decisions based on intelligence analyses. It has been recognized for years that we are far better at collecting, transmitting, storing, and retrieving intelligence information than we have been at drawing defensible and persuasive conclusions from it. 261 This gap between collection and analytic methods has been brought into particularly sharp focus by the tragic events that took place on September 11, 2001 involving the terrorist attacks on the World Trade Center and the Pentagon. Intelligence analysts face extremely complex inferential tasks involving the generation and analysis of true masses of evidence of every conceivable kind. In short, intelligence analysts face the task of trying to make sense out of masses of evidence that, on the surface at least, may not appear to make any sense. The NSA along with its counterparts in Canada, UK, Australia, and New Zealand uses a worldwide network to intercept messages. How much raw data these intercept facilities haul in every day is classified. But some observers speculate it is comparable to all the information in all the books in the Library of Congress! Glenn Borpette, Senior Editor IEEE Spectrum Online, Jan 02 www.spectrum.lieee.org Everyone agrees that the collection of relevant and credible intelligence evidence of a variety of sorts is vitally necessary. But collection by itself, however necessary, is not sufficient; we must become more adept at drawing imaginative, defensible and persuasive conclusions from it. It would not be stretching the point to say that our future may depend upon your ability to productively generate imaginative and plausible hypotheses and then to be able to defend hypotheses you believe are most likely based on coherent 261 See: Schum, D., (University Press of America, Lanham, MD, 1987), Vol I, pages 4 and 470-471.
and persuasive arguments you construct from the evidence at hand. These are the core activities of intelligence analysis — the “science of analysis,” if you will — discovery, proof, and choice. As we all know, failures of intelligence analysis are often widely advertised while successes are not made public, and often cannot be. The truth is that analysts are routinely asked to perform tasks for which they have received little if any tutoring. Conventional courses in logic, probability, and statistics do not prepare a person for the task of drawing conclusions based on masses of evidence whose items suggest many, often complex and interrelated, lines of arguments on hypotheses of interest. The evidence of interest to intelligence analysts usually concerns events that are unique, singular, or one-of-a-kind and are thus not replicable or repeatable. This means that there are rarely any useful or relevant statistical records available to draw upon in making inferences about the capabilities and intentions of potential or real adversaries. We had no existing statistical records regarding the intentions of foreigners who showed up in our civilian flying schools wishing only to learn how to steer multiengine aircraft and not how to perform takeoffs or landings. Lacking existing statistical records to draw upon, we must generate new information by inquiry, the asking of questions. Skill in asking productive questions is as vitally necessary in intelligence analysis as it is in any other situation in which discovery and investigation are necessary. Effective intelligence analysis rests upon mixtures of both imaginative and critical reasoning. Intelligence analysts whose work has been exemplary have almost certainly acquired their skill through years of experience and, perhaps, through often-painful trial and error experiences. Given the need for accurate and timely intelligence analyses, whose urgency is evident in light of current events, we cannot afford to have intelligence analysis learned just on the basis of many trial and error experiences. But where are the skills of marshaling evidence, analyzing evidence, and the construction of persuasive arguments in support of intelligence estimates taught? Do we assume that such analysis is something that can be done instinctively? Or do we think that collecting and cataloging evidence is more specialized than analyzing evidence? I start from the premise that such assumptions are false. Basic skills in marshaling and analyzing evidence are important and teachable skills that are largely neglected in our intelligence analysis education. Just as you cannot do very much carpentry with your bare hands, there is not much thinking you can do with your bare brain. Bo Dahlbom and Lars-Erik Janlert As cited by Dennitt in “Kinds of Minds,” 1996 Fortunately, there are methods for marshaling masses of evidence and for constructing complex arguments that are very useful and that may at least reduce the number of trial and error experiences that so often accompany learning the business of accurate and timely intelligence analyses. Some of these methods have been in existence for decades, but have not been given the attention they deserve. The exception is in the field 126