I__. - International Military Testing Association

overall predictive efficiency, (i.e., the reduction in r between
TI and PTI). The process continues until all data are combined
into a single equation. Analysis of the r losses at each stage
allows identification of the fewest number of regression
equations that can accurately generate FTI values across all
AFSs. In order to measure how well each set of weights would
reproduce an ATO, the weights were used to classify tasks into
the "A-D" categories. PTI values were classified into importance
categories of A through D using a procedure identical to the one
for TI values. Classification accuracy (CA) was measured by
computing the table and formula shown in Figure 1.
Predicted Ch0~ltlcatlon
A B C D
A F 11 F 11 F 1s F 14 Rl
B
Actual F 21 F 22 F 2s F 24 R2
Cluoiflcatlon
c F91 F SP F 93
F R3
94
D F 41 F 44 F,, F 44 Fk
Cl c2 cs c4 N
F;; ir the frequency in the iJ%ll
Figure 1. Classification Table and Formula
CA has been weighted such that misclassifications result in
larger penalties as the V'distance18 between predicted
classification and correct classification becomes greater. This
weighting strategy is reasonable, in that testing importance
differences associated with categories in the table become
greater as the 8*distance1V between the categories increases. The
range of CA values is 0% (every classification has maximum
distance from the correct classification) to 100% (every
classification is correct).
RESULTS
The r's using weights from the cross-validation samples
ranged from .44 (908X0) to .92 (914X0) and the r's using weights
from the validation samples ranged from .42 (566X0) to .91
(121X0). Therefore, there is great amount of variability among
the AFSs in the ability of a linear function of the five
predictors to account for the variance in TI. Because the
shrinkage in r using the weights from the validation and crossvalidation
samples was nonsignificant (a(=.O5) across all AFSs,
the validation and cross-validation samples were combined for
subsequent analyses.
313
_