I__. - International Military Testing Association

Psychometric Properties of NOAE! Predictors
As a result of the inability of some NOAB predictors to
provide criterion prediction, an evaluation of the psychometric
properties of the NOAB exercises was conducted using two
approaches.
Factor analvtic approach. The 30 dimensions measured by the
five NOAB exercises were submitted to a principal components
analysis (varimax rotation) which produced a seven-factor solution
shown in Table 2. As illustrated in Table 2, conceptually
independent dimensions underlying the first four NOAB exercises .'
loaded on exercise factors rather than on factors with conceptually
similar dimensions. It appears that these four exercises are
producing global measures of overall performance on each exercise
and not measuring exercise dimensions, thereby raising doubt about
the construct validity of the dimension ratings that comprise each
of the exercises. Candidates' scores on these exercises may be
more attributable to the procedures followed by the NOAB than the
candidates' abilities with respect to the dimensions the four
exercises are supposed to be measuring. .Table 2 shows that the
file review was the only NOAB exercise that appeared as a
multidimensional construct (it measures three different constructs
-- personal background, military experience, and intelligence).
In addition, the file review was the only NOAB measure that
predicted MAPS training performance. The fact that the dimensions
underlying the file review loaded on factors with other
conceptually similar dimensions and did not simply load on a file
review factor provides evidence of construct validity for the
dimension ratings comprising the file review score and may account
for the file review's success as an NOAB predictor.
Multitrait-multimethod matrix annroach. To investigate
further the notion that the interview, leadership stands,
conducting officer's assessment and leaderless group discussions
are actually producing one global measure for each exercise without
regard to the dimensions contained in the exercise, the
correlations of conceptually similar across-exercise dimensions
(similar dimensions measured by different selection exercises) were
evaluated using the method described by Campbell and Fiske (1959).
As reported in Bradley (19901, the correlations between
conceptually similar across-exercise correlations were lower than
correlations between conceptually independent within-exercise
dimensions, thereby lending further support to the notion that
method variance is contaminating the measurement of NOAB exercise
dimensions.
Discussion
The results of this validation research can be summarized as
follows: (a) the file review is the only NOAB measure that predicts
both training criteria; (b) the leadership stand assessment
265