I__. - International Military Testing Association

An AT0 is organized by duties and tasks within duties. All
tasks on the TI survey are listed under the appropriate duty.
The TI values are used to weight the duties and tasks. To
accomplish this weighting, the TI values for each task are
squared and summed within a duty. The weight for each duty is
the sum of the squared TI values across all tasks within the duty
divided by the sum of the squared TI values across all duties.
These weights are the percentages of test questions to be
selected to cover the required knowledges to successfully perform
the tasks within each duty.
The TI value of each task within a duty is reflected by a
letter from A to D. Tasks are designated as r'AV' tasks if their -.
TI values are at least one standard deviation above the mean of
the TI values or if their TI values are at least 6.0. Similarly,
tasks are designated as 'ID" tasks if their TI values are more
than one standard deviation below the mean of the TI values:
however, all tasks with TI values of at least 4.00 are designated
as qVC'* tasks. Of the remaining tasks, the upper 50% are
designated "B" tasks and the lower 50% are designated "C" tasks.
SMEs are required to write at least one item to test the job
knowledge required for every '@A" task and to write no more than
three items for a single task. Procedures are available to
override these restrictions; however, they require written
justification. Items can be written on "D1@ tasks only with the
group facilitator's approval.
PROCEDURE
Tasks for each of 26 AFSs for which testing importance
indices were available (914X0, 753X0, 423X3, 791X0, 423X4, 792X1,
915X0, 908X0, 392X0, 231X2, 542X2, 674X0, 552X0, 324X0, 542X1,
427X3, 321XlE, 112X0, 121X0, 274X0, 321XlG, 241X0, 431X0, 275X0,
566X0, and 231X0) were randomly divided into two samples--one
sample designated the Validation sample" and the other sample
designated the "cross-validation sample." First, the IIAtt tasks
were randomly split between the two samples, such that each
sample contained approximately an equal number of l*AVV tasks. The
IIB, II IICII and I'D“ tasks were split between the two samples in the
same manner. Regression equations were computed separately for
each validation and cross-validation sample with TI as the
criterion and PMP, PTM, AG, TD, and TE as the predictor
variables. The two sets of regression weights computed for each
AFS were applied to the predictor scores for the cross-validation
sample to generate predicted testing importance (PTI) values.
The predictive efficiency of each set of weights can be measured
by the Pearson coefficient of correlation (r) between TI and PTI.
If the shrinkage in r using the two sets of weights on the
cross-validation sample is statistically nonsignificant (Walker t
Lev, 1953), then the data for both samples can be combined for a
hierarchical clustering analysis. In this procedure, the number
Of regression equations is reduced by one at each stage of the
clustering by combining AFSS into groups and combining their
corresponding regression data. The two most similar groups are
combined at each stage, as measured by the resulting loss of
312
,
i