2003 IMTA Proceedings - International Military Testing Association

It can be seen that for this AIM scale, the OAM procedure differentiated stayers and leavers
to a moderate degree. For example, for this scale, at a 20% false positive rate, 33% of
leavers were correctly identified. Note that because the AIM is currently used operationally
to predict attrition, we do not present results that can identify which AIM content scales
worked best.
The results also indicated that a LR composite provided the highest hit rates among
the seven decision variables. It correctly identified 22% of stayers at a 10% false positive
rate, 35% of stayers at a 20% false positive rate, 47% at 30%, 56% at 40% and 65% at
50%. The success of the LR composite indicated that AIM content scales provided
incremental validity in the prediction of attrition and, thus, should be used collectively.
It is important to note that the use of OAM methodology provided an improvement
over the current application of the Adaptability score in predicting attrition. For instance, at
about at 20 percent false positive rate, the current adaptability score yields 27 percent
correct identification rate of those who leave the service, while the OAM composite yields
a 33 percent correct identification rate. A graphical comparison of ROC curves for the two
identification procedures (see Fig. 3), showed that OAM method performed better than
Adaptability score at every level of the false positive rate. Thus, based on these results, we
recommend using the OAM-based LR statistic to predict the likelihood of attrition with
AIM instead of the Adaptability composite.
Fig. 3. ROCs for OAM and Adaptability Composites
Hits
100%
80%
60%
40%
20%
ROCs for OAM and Adaptability Composites
0%
0% 20% 40% 60% 80% 100%
False Positives
321
45 th Ref Adapt OAM Composite
Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003