2003 IMTA Proceedings - International Military Testing Association

We also have under way a predictive validation study to evaluate the validity of the test
battery for predicting the subsequent performance of recruiters in a testing setting that is more
similar to an actual operational environment for the test administration. Non-commissioned
officers (NCOs) entering the Recruiting and Retention School (RRS) are being administered the
test battery, and we have now followed up on a sample of 1,466 of these recruiters. Attrition data
from the RRS and production data (i.e., number of recruits brought into the Army per month)
were available for these recruiters, and we also present initial predictive validation results using
these indicators as criteria.
Long-term goals for the project are to establish a standard screening process for NCO
candidates for a recruiting assignment prior to their being accepted into the RRS. NCOs scoring
well on the battery might be encouraged to volunteer for recruiting. An even broader goal is to
develop a classification test battery to target placement into other possible second-stage jobs
(e.g., drill instructor). In one scenario, this classification battery would be administered routinely
to NCOs at the beginning of their second tour, and predicted performance scores would be
generated for each target job. Then, NCOs could be counseled about which second-stage job(s)
suit them best.
THE CONCURRENT VALIDATION STUDY
We first conducted a job analysis of the Army recruiter military occupational specialty
(MOS). This analysis had two purposes: to identify the recruiter performance requirements and
thus to suggest performance measures that might be used as criteria in the validation study; and
to identify candidate predictor tests for the validation research.
Criteria
. The job analysis suggested that production rates (as mentioned, number of prospects
brought into the Army per unit time), and peer and supervisory ratings of job performance on the
main dimensions of recruiter performance would provide relevant criteria. We also decided to
develop a situational judgment test to measure problem-solving, judgment, and decision-making
skills important in recruiting. More details on the criterion measures will be provided in a
moment.
Predictors
Predictor measures for the test battery included: (1) The Army Research Institute’s
Background Information Questionnaire (BIQ), with eight scales including “natural” leader,
social perceptiveness, and interpersonal skills; (2) The Army Research Institute’s Assessment of
Individual Motivation (AIM), with six scales including work orientation, leadership, and
adjustment; (3) The Sales Achievement Profile (SAP), with 21 scales including validity scales,
sales success, motivation and achievement, work strengths, interpersonal strengths, and inner
resources; (4) The Emotional Quotient Inventory (EQI), with 15 scales including intrapersonal,
399
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003