learning with professionals - Higgins Counterterrorism Research ...


learning with professionals - Higgins Counterterrorism Research ...

Regardless of its use, “no piece of evidence can be used in the state in which it is

found.” 86 Whenever an analyst considers new evidence, the critical mind must follow a

systematic interrogatory:

Are these data genuine?

How do I know?

What does this evidence state or imply?

Who is its author or maker and what degree of access does he have? 87

What is the relation in time and space between the author and the information,

overt or implied, that is conveyed by the information?

How does the statement compare with other statements on the same point?

What do we know independently about the author and his credibility? 88

The product of critical inquiry is a judgment about the credibility and reliability of the

evidence and its source. 89 Credibility (a measure of confidence) is only partly based on

reliability (a measure of consistency), since past performance is never perfectly repeated.

Pertinent data may not be available when an analyst begins to verify a piece of evidence.

Under these circumstances, common sense and educated guesswork may be the

only means of answering the questions asked in the evaluation of evidence. The specific

basis of the answers is important and must be recorded. In this way, resort to intuition as

the basis for decisionmaking is avoided. The key to the process is not that all of the

answers to the inquiry are based on fully verified facts; rather, that the process is systematic

and each answer is documented.

Systematic Analytical Method. When an analyst systematically verifies data, authenticates

a source, or accepts the likelihood of a hypothesis, important benefits ensue if the

details of the process are recorded. Aside from serving as a paper trail that proves an evaluation

was accomplished, documentation will hold crucially important information: the degree

of rigor involved, what was known and unknown, what the analyst had confidence or doubts

in, and a level of plausibility or acceptability of the evidence. A record of the manner in

which the analyst evaluated the evidence also exposes the analyst’s assumptions and values

for consistent application when other evidence arises. Finally, these records can be available

for periodic review and reassessment by both the analyst and others who need to understand

the analyst’s line of reasoning to evaluate the acceptability of the conclusions.

86 Barzun and Graff, 156.

87 There are four levels of verifiability: personal observations, reports, inferences, and assumptions. These

levels correspond to the directness of the source to the information being reported. Karl Albrecht, Brain Power:

Learn to Improve Your Thinking Skills (Englewood Cliffs, New Jersey: Prentice-Hall, 1980), 121.

88 Barzun and Graff, 157-158.

89 Credibility is a level of confidence in someone’s or something’s credit-worthiness. Reliability refers to the

repeatability, including intrasubjective replicability, of observation. Suppose a technical surveillance system or

human source produces approximately the same set of responses on repeated trials (and with different analysts

or debriefers), we can say that the observational technique/source has high reliability, regardless of the actual

validity of the findings. Pertti J. Pelto, Anthropological Research: The Structure of Inquiry (New York: Harper

and Row, 1970), 42.


More magazines by this user
Similar magazines