Analytic Culture in the U.S. Intelligence Community (PDF) - CIA

RECOMMENDATIONS
ligence analysis; rather, each type of task will have an analytic method that is
best suited to accomplishing it in an efficient and effective manner.
Developing these metrics is no small task. It is a job that will require
numerous researchers and research programs within, or with access to, the
Intelligence Community. These programs will need formal relationships with
human resource departments, analytic divisions, organizational leadership,
and developers of training and technology interventions in order to have a
positive effect on analytic performance.
Research Programs
The results of this research indicate that the Intelligence Community needs
to commit itself to performance research that is rigorous, valid (in that it measures
what it proposes to measure), and replicable (in that the method is sufficiently
transparent that anyone can repeat it). Within some intelligence
organizations, this has been an ongoing process. The problem is that most of
the internal research has concentrated on historical case studies and the development
of technological innovations. What is missing is focused study of
human performance within the analytic components of the Intelligence Community.
Questions about the psychology and basic cognitive aptitude of intelligence
analysts, the effectiveness of any analytic method, the effectiveness of
training interventions, group processes versus individual processes, environmental
conditions, and cultural-organizational effects need to be addressed.
This effort will require commitment. Researchers will have to be brought
into the Intelligence Community, facilities will have to be dedicated to
researching analytic performance, expert analysts will have to give some percentage
of their time to participating in research studies, managers and supervisors
will have to dedicate time and resources to tracking analytic
performance within their departments, human resource staffs will have to dedicate
time and resources to developing a performance repository, and there
will have to be formal interaction between researchers and the community.
Analytic Performance Research. In the previous section, I discussed the
need for analytic standards as part of the Performance Improvement Infrastructure.
In terms of a research program, this will require, as a first step, the
collection of baseline analytic performance data and a clear and measurable
description of ideal analytic behavior. Next, there should be a determined
effort by human performance researchers to develop, test, and validate analytic
performance metrics and measurement systems. This will be a lengthy
process. The accuracy and surprise measures suggested in this text require
large historical and comparative data sets and are cumbersome and time consuming
to perform. Conducting behavioral, cognitive, and linguistic task
111