2003 IMTA Proceedings - International Military Testing Association

Table 1: Misclassification rates for five classification trees
“False positives”
(misclassification of
Number of
terminal nodes stayers, percentages)
3 31.14% 45.32%
6 34.23% 48.19%
7 33.64% 47.40%
11 33.40% 47.13%
18 32.09% 45.68%
“Hits”
(correct classification of
leavers, percentages)
The first and third of these trees are depicted in Figures 1 and 2, respectively (where left
branches indicate a “yes” response, while right branches indicate a “no” response to the decision
rule in the parent node). For example, Figure 1 shows that the root (i.e., initial) node splits the
sample on the basis of Scale D scores; individuals with scores on Scale D of less than 8.5 are
predicted to be leavers and individuals with scores greater than 8.5 are branched to another node.
In this internal node, individuals with relatively high scores on Scale D (i.e., greater than 8.5) but
low scores on Scale B (less than 14.89) are predicted to leave. It is only individuals with high
scores on both Scales D and B that are predicted to be stayers.
Figure 1: Classification tree with 3 terminal nodes
CART also rank-orders the relative importance of the predictor variables. In this
analysis, CART identified the two best predictors as Scale D and B. Scales A, E, and C played a
smaller role in these classification trees, whereas Scale F played a nearly insignificant role. In
comparing the performance of these trees to the Adaptability Composite, we turn to a receiver
operating characteristic (ROC; Figure 3) curve depicting the relative hit and false positive rates
associated with separate cut-off scores for the Adaptability Composite and the classification tree
with 7 terminal nodes. A similar pattern was found in comparing this tree against results from
logistic regression, where all six of the content scales served as predictors.
313
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003