Technical Report - International Military Testing Association

ecause we do not like the criteria is confusing the issue. Hoffmann’s
concern rfth ambiguity is relevant only when the content vaLidity of a
test is the primary constd.eratFon. Eve,1 80, tke customary item acreesing
and selection in the constructicn of teats insurea a reaeonably satiafactory
degree of content valfdity. This io particularly true in the
case of our military tests which are designed to cover specific job areas.
What about Hoffa%M'S cLaim that the “deep” examinee fs penalized
by multipLe-choice iteM? What he seems to be saying is that the very
dull examinee will fail the itea more frequently than the superficially
bright examinee, but that the exceptLonally bright examinec vi11 have
more trouble with the same item than the superficially bright examinee,
Naturally, the brightest examinees are more apt to discover ambiguities
than are other students. But Hoffmann’s ideas about the consequences of
their perceptiveness do not hold up.
To get back to Hoffmann’s assumptions, he states that tests,“... take
account of only the choice of answer and not of quality of thought that
lead to the choice,” and “They neglect skill and dtsciplfned expression.”
This attribute he catls “quality of thought I’ is not defined, nor is any
reliable and valid way of measuring it suggested. One mfght assume that
Hoffmann would advocate use of the essay examination as a measure of his
quality of thought. Such is not the case, for he outlines no less than
four very convincing arguments why the essay test should not be used for
that purpose. (L. Difficult to choose a topic fair to a% 2. difficult
to determine whether the essay is actually revelant to the question; 3.
difficult to overcome the problem of negatfvc halo due to poor handwriting,
spelling, etc. ; 4. difficulty to maintain consistency within and between
scores.) I often hear this quality of thought argument twisted around to
say that just because a man scores high on the test does not necessarily
mean that he can do the job. Most generaLly, those advancing this
argument mean that the man will not do the job. While it may be true that
a high score on an achievement or’ job knowledge type test does not
automatically indicate a high degree of motivstion, it is eqtially true
that the Low scorer cannot perform the fob, that is, he does not have
the knowledge to perform regardless of hie motivation. Of course this
depends on test validity and test purpose. It is up to us to define
the purposes to which oar instruments may be put and to determine the
validity, for various purposea, of these instruments.
At this point we might conclude that Hoffmann does not like any kind
of testing. We would probably be correct. He criticizes objective tests;
yet he leaves no alternatives for mensurement. When he argues that &he
results of an educational testing service st*)qy showing an essay teat
was less good than an objective examinstion are silly and could not
possibly have been obtained, he is simply confusing content with predfctive
or concurrent validity.
176
,. ^ .._ -.. .__.-
.