Analytic Culture in the U.S. Intelligence Community (PDF) - CIA

SURVEY METHODOLOGY
might merely serve to emphasize the small size of Intelligence Community. In
any case, this approach to sampling may help to ameliorate the sampling bias
inherent in qualitative research.
In addition to semi-structured interviews, both direct and participant observation
data collection methods were employed. The direct observation method
involved watching Intelligence Community analysts perform their tasks in
both actual and training environments, recording the physical and verbal interactions
they had with one another, and observing the steps used to create intelligence
products. Direct observation occurred over the course of two years by
observing 325 individual analysts and teams of analysts performing their specific
tasks. The data collected from observing the 325 analysts were not
included in the semi-structured interview data because I did not use the formal
semi-structured interview process to structure those interactions. These observational
data were recorded separately in field notes and used for triangulating
the findings from the interviews.
The participant observation method is employed to give the researcher a
“first-person” understanding of the context and nuances associated with a task
and the culture in which that task occurs. Although the researcher possesses
only an approximation of the knowledge and understanding of the actual practitioners
of the task and their culture, this “first-person” perspective can lead
the researcher to new insights and new hypotheses.
During this study, the participant observation was conducted during analytic
production cycles, scenario development, and red cell exercises. This
included monitoring my own analytic strategies, the analytic strategies of others
as diagramed or verbalized, the physical and verbal social interactions
among the participants, the environment in which the tasks occurred, and the
steps used to create a final intelligence product. These data, along with notes
on social dynamics, taboos, and social power, were recorded in field notes and
created a separate data source for triangulation.
With modern anthropology, these data normally would be captured on film,
audiotape, or in some digital format. Due to the security requirements of the
Intelligence Community, however, the data were captured only in the written
form of field notes. As is the case with the field notes, the identity of the interview
participants will not be disclosed. This is in keeping with both the security
practices of the Intelligence Community and the professional standards
described in the American Anthropological Association Statement on the
Confidentiality of Field Notes. 7
The data from the interviews were analyzed using a method called interpretational
analysis. 8 This approach included segmenting the interview data into
7
American Anthropological Association, Statement on the Confidentiality of Field Notes.
123