2003 IMTA Proceedings - International Military Testing Association

156
Does Military Personnel Job Performance in a Digitized Future Force Require
Changes in the ASVAB: A comparison of a dynamic/interactive computerized
test battery with the ASVAB in predicting training and job performance among
airmen and sailors.
Ray Morath, Brian Cronin, & Mike Heil,
Caliber Associates
In the late 1990’s a team or researchers developed, as part of an award-winning project (SIOP Scott-Meyers
Professional Practices Award, 1999), a battery of dynamic, interactive/computerized tests that is currently
being used to select air traffic controllers; This battery of tests is known as the air traffic selection and
training (AT-SAT). An empirical validation study using a concurrent sample of over 1,000 job incumbents
found that the AT-SAT to have possess high predictive validity as well as high face validity. The AT-SAT
was designed to measure the cognitive, perceptual, and psychomotor abilities critical to the air traffic control
job, however, it has been found that many of these same cognitive, perceptual, and psychomotor abilities
have also been identified as important for military officer, non-commissioned office, and enlisted personnel
performance (Campbell, Knapp, & Heffner, 2002; Horey, Cronin, Morath, Franks, Cassella, & Fallesen, in
press; Noble & Fallesen, 2000; Rumsey, 1995).
Our team of researchers has recently created a parallel form of the original AT-SAT and is conducting an
equating study to ensure that the new form is of equivalent difficulty and complexity and measures the same
performance domains as the original battery. This equating study involves collecting data from
approximately 1,500 Air Force and Navy personnel who have recently completed boot camp and are about to
enter technical training programs for their assigned MOS. Our study will compare airmen and sailor scores
on the AT-SAT battery to their scores on the ASVAB and will also investigate the ability of the AT-SAT to
predict variability in training and job performance that is unique from that already predicted by the ASVAB.
Our paper will present the research methodology for creating and validating the AT-SAT and will present
data regarding the correlations of the various sub-tests with multiple dimensions of air traffic controller
performance. We will also present data linking the AT-SAT to individual cognitive, perceptual, and
psychomotor abilities required by air traffic controllers and discuss how many of these abilities are required
not only within various technical jobs, but also across officers, NCOs, and enlisted personnel. Finally, we
will discuss the influence of digitization on military performance requirements—specifically within the areas
of technical training performance and tactical and technical performance, and the need for new, dynamic and
interactive methods, such as the AT-SAT, of measuring the abilities associated with these changing
requirements.
Campbell, R. C., Knapp, D. J., & Heffner, S. T. (2002). Selection for leadership: Transforming NCO
promotion. (ARI Special Report 52). Alexandria, VA: US Army Research Institute for the Behavioral and
Social Sciences.
Horey, J., Cronin, B., Morath, R., Franks, W., Cassella, R., & Fallesen, J. (in press). Army Training and
Leader Development Panel Consolidation Phase: U.S. Army Future Leadership Requirement Study.
Prepared for the U.S. Army Research Institute under contract (DASW01-98D0049). Caliber Associates:
Fairfax, VA.
Noble, S. A., & Fallesen, J. J. (2000). Identifying conceptual skills of future battle commanders. (ARI
Technical Report 1099) Alexandria, VA: US Army Research Institute for the Behavioral and Social
Sciences.
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003