2003 IMTA Proceedings - International Military Testing Association

homogeneity of work activities within each occupation. Finally, O*Net’s Occupational Skills
taxonomy was used to characterize the specific requirements of particular NIMA occupations.
METHODOLOGY
NIMA’s new personnel system was developed during a four-step process. The first step
involved grouping legacy jobs into a parsimonious occupational structure and collecting data to
statistically examine the quality and comprehensiveness of these groupings. The second step
involved developing work roles within each occupation, again using empirical data to examine
quality and comprehensiveness. The third step involved populating the newly-developed skills
database with employee competency data. The fourth and final step involved periodically
reviewing and updating the work roles to account for new developments in the ever-changing
world of geospatial intelligence analysis.
Grouping Legacy Jobs into Occupations
Our initial goal was to create 20 to 30 broad occupations of employees who used similar
skills in performing similar activities. We first assembled a comprehensive list of the Agency’s
legacy position descriptions. A panel of Industrial and Organizational Psychologists then
grouped the position descriptions into a smaller, yet comprehensive list of composite titles. We
then created a description for each composite which summarized the aggregate duties and
principal differences among the positions within the composite. Subject Matter Experts (SMEs)
then reviewed the descriptions and eliminated or combined composites to reflect near-term
outsourcing or other changes that were expected to occur within NIMA. The result was a set of
approximately 125 composite descriptions that summarized all of the work performed at NIMA.
We used statistical data to guide our decisions regarding the most appropriate
occupational structure. To statistically group the composite positions into a smaller number of
relatively broad occupations, it was necessary to create profiles on a common set of descriptors.
We collected SME ratings using O*Net BCFSs and cluster analyzed the profile ratings.
Hierarchical agglomerative procedures were used to iteratively build increasingly larger clusters
until all of the composites had been clustered together into a single group (Everitt, 1993;
Kaufman & Rousseeuw, 1990). At the end, a dendogram graphically displayed the order in
which the composites were combined and indicated the magnitude of the differences between the
composites being combined at each step. We reviewed each dendogram, along with the matrix of
distances between composites. Based on this information, we identified approximately 30
clusters – or potential occupations – that appeared to be both statistically and practically viable.
We then presented the results to NIMA managers and other critical stakeholders to elicit their
reactions, concerns, and approval. The results were considered along with a host of practical and
political organizational factors to arrive at a final structure of about 25 broad occupations.
Defining Work Roles within Occupations
We began to define work roles within each occupation by assembling panels of SMEs
from each occupation to offer guidance regarding the preliminary work roles. These panels
identified meaningful distinctions among the jobs within their occupation, and developed
preliminary titles and general descriptions for each work role. At the conclusion of the SME
panels, each occupation had a defined set of work roles, with a total of about 200 preliminary
273
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003