Analytic Culture in the U.S. Intelligence Community (PDF) - CIA

CHAPTER ELEVEN
Q-Sort Methodology
Observations always involve theory.
Edwin Hubble 1
As described earlier, this work reflects triangulation of the data derived
from the literature Q-sort, interview responses, and observations. 2 The data
include 489 interviews, direct and participant observation of 325 analysts performing
their jobs, participation in a variety of analytic tasks, and focus
groups conducted to generate the taxonomy of variables that guided this study.
The first Q-sort of the data was aggregated according to the function of
each intelligence organization, as listed in Table 1. The data were then analyzed
to determine response context according to job type and to develop variable
categories.
The organizational Q-sort generated the broad variable groupings used to
create the second Q-sort parameters. The variable categories that emerged
during the interpretive analysis of the first Q-sort of the data were compiled
again, and a second Q-sort was performed based on those categories. The data
was then aggregated according to categorical or variable groupings of the second
Q-sort, Table 2.
The use of two separate Q-sort strategies generated the variables and then
de-contextualized the data in order to find consistent trends throughout the
Intelligence Community. That is, this strategy resulted in broad categories of
findings that apply across many agencies. In those cases where interview and
1
Edwin Hubble discovered the first evidence to support the Big Bang theory that the universe is
expanding and that the Milky Way is not the only galaxy in the universe. He also developed the
Hubble Galaxy Classification System and Hubble’s Law (the farther away a galaxy is from Earth,
the faster its motion away from Earth). Edwin Hubble, The Realm of the Nebulae.
2
William Stephenson, The Study of Behavior: Q-Technique and its Methodology.
127