Technical Report - International Military Testing Association

f
Ih
i
i
Hoffmann attempts to document his reasoning -- and it in mere11
U’arm-chairing” -- that multipie-choice test qucctitions are by neture
ambiguous by citing Pample items from test manuals or more frequently
by attacking illustrative items of his 0.m. A Hoffmann-type item C.W
quickly illustrate the kind of rumination which forms the main content
of his attack,
i
.
I .
Watt are the colors of the American flag?”
(A) red, white, and blue
i
c
, 0)
((3
say
neither A nor B
i The superficial examinee quickly selects answer “A,” red, white, and blue,
and goes to the next question. The “deep” examinee begins to scrutinize
and analyze the alternatives; he thinks, “A” is correct under some
conditions, but “B,” gray, Is correct under some conditions too -- twilight,
poor illumination, total color blindness. Both “A” and “B” could be correct
under some conditions, but the other alternative, “neither A nor B,” could
not be correct. Supposedly, ha wonders if the questfon is a trick, if
the test wrftcr is malicious or just plain ignorant, and what was really
wanted. Finally in desperation, t,e throvs up his hands and says, llThie
is an absurd test nnd 1 don’t see how any intelligent person can be asked
to take it seriouslylf’ And 1 have heard just that in the field on more
than one occassion.
This charge of ambiguity is one of most serious consequence to us
and one on which we are most vunerable unless we consider the fundamental
distinctions among the purposes of test and test items. I am referring
to the distinction between tests which are used as criteria ant. tests
which are u6eu as predictors. - -
Criterion tests -- for example, achievement
tests for comparing the effects of dffferent methods of teaching -may
be open to Hoffmann’s criticisms. But his objections are lrreievant
in the case of predictive tests -- for example, tests to select the most
promising job applicant or the best qualified soldier for a special assignment.
Basically a criterion test must be content valid -- that is,
we must be able to defend the test on rational and logicalgrounds. All
that fs required of predictive tests is that they sucessfully predict
future performance.
Now where does this leave those of us who use our tests for both
purposes? Naturally we must attempt to demonstrate that vdt tests have
both content validity and predictive or concurrent validity. It appears
chet Hoffmann has never heard of empirical validation, but he manages
to marshal1 a variety of defense against the evidence. He particularly
attacks the criterion sgainst which we validate our tests. However) the
value of a criterion is a very different question frocn the problem of
whether a test can predict that criterion. Criticizing the tests simply
175
t