2003 IMTA Proceedings - International Military Testing Association

402
THE PREDICTIVE VALIDITY RESEARCH
The recruiter screening battery has considerable potential for identifying NCOs likely to
be successful recruiters. However, we believe it is highly important to evaluate, as well, the
validity of the battery over time in a predictive validation design. As mentioned, this research is
underway at the RRS, where incoming students at the school are being administered the test. In
fact, 1700+ recruiters who completed the test battery, recently called the NCO Leadership Skills
Inventory or NLSI, have now progressed through the RRS (although 10.9% dropped out during
training) and 1,466 have at least five months production data available for the predictive
validation analyses. Below we describe that research.
Criteria
We used two criteria in the predictive work. Attrition from the RRS was one criterion,
and production rates corrected for territorial differences, as in the concurrent study, was the other
criterion. We included recruiters in the study only if they had five or more months of production
data. The reliability of the corrected production index was .64.
Predictors
As mentioned previously, the test battery now included the BIQ and the AIM, and is
referred to as the NLSI.
Sample
More than 1700 NCOs entering the RRS were administered the NLSI. Approximately
160 of these NCOs were dismissed from or otherwise failed to complete training. A total of 1466
recruiters graduated from the RRS and had five or more months of production data.
Results
The predictive validity of the NLSI was also promising, although at this point the criteria
are somewhat limited. First, the attrition results showed that when NLSI scores are divided into
quartiles, attrition was respectively, from highest to lowest scorers, 6.5%, 8%, 10.5%, and
18.8%. Even more dramatic, the bottom 5% of NLSI scorers has an attrition rate of 36%,
whereas the rest of the sample attrited at a 9% rate.
Second, the production rate results indicated that recruiters who score in the top 25% on
the NLSI brought in 1.10 recruits on average. For the second highest quartile the number was
1.06, 1.03 for the third highest quartile, and .91 for the lowest quartile. At the extreme bottom of
the NLSI distribution, the lowest 5% bring in .75 recruits, whereas the rest of the sample has
1.05 as an average. For comparison purposes, the BIQ-AIM composite cross-validity against
production in the concurrent study was .22. The corresponding validity coefficient in the
predictive study was .15.
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003