2003 IMTA Proceedings - International Military Testing Association

Abstract
Pilot Selection in the Australian Defence Force:
AUSBAT Validation
Dr. Alan Twomey
Major Damian O'Keefe
Psychology Research and Technology group
Department of Defence, Australia
Using a concurrent validation design, this study applied regression analytical techniques to develop
a statistical model for improving the prediction of outcomes at Basic Flying Training in the
Australian Defence Force (ADF). In particular it sought to evaluate the relative contribution made
by the Australian Basic Abilities Test (AUSBAT) battery of tests compared to the existing selection
battery, relevant biographical variables, some alternative trial tests, and a flight screening program
to predicting basic pilot training outcomes. Hierarchical regression analysis found that 46% of the
variance in overall sortie average rating scores could be predicted with Flight screening accounting
for most of this variance followed by one of the AUSBAT tests (Pursuit B). Cutoffs identified from
the regression equation enabled three groupings of trainees to be developed with failure rates in the
order of 33%, 19% and 0% respectively.
Introduction
This paper describes an attempt to improve the prediction of training outcomes at the Australian
Defence Force (ADF) Basic Flying Training school. In the process it reports the latest phase in the
development and evaluation of the Australian Basic Abilities Tests (AUSBAT) battery as a tool for
military pilot selection.
AUSBAT
The AUSBAT tests are computer generated tests delivered via a standard desktop PC utilising
specialised joy sticks. The theoretical underpinnings, early development and descriptions of the
tests have been reported by Bongers, S and Pei, J., (2001).
Since its conversion from a DOS environment to a Windows platform in 1999-2000, AUSBAT has
undergone systematic evaluation. Completed phases include the following:
a. Specification of standardised delivery parameters for each of the tests (Pei J., 2002)
now incorporated into technical documentation;
b. Development of standard scoring systems for the tests (O'Keefe D., 2002a);
c. Investigation of the construct validity of the tests (O'Keefe D., 2002a, Pei J., 2003);
and
d. initial concurrent validity studies against basic training outcomes (O'Keefe D.,
2002b).
Today, the AUSBAT battery comprises nine discrete tasks grouped in various combinations,
orientations, and difficulty levels, into fourteen tests, yielding twenty-eight measures. These
measures comprise accuracy scores (e.g. time spent in target area, number of correct answers) and
error scores (e.g. distance away from target area, orientation error of test objects in relation to
reference targets, and number wrong). Standard deviation scores are also calculated for 11
psychomotor accuracy and error scores.
461
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003