Technical Report - International Military Testing Association

.___ ---.-.-a ---._- ~.-~“..CI.~.--..w.‘r -.--.-.-^_. -. ._ .
.’
!
)
he handled his contacts, how he cooperated vith the home office, everything
we could about him. Then WC would ask, “gow about one of your
poor ones? Who is.the next one you would drop if you had to cut iour
work force? Haw do you know he is that bad?” We found enormous overlap
in observable performance of the better as compared with the poorer men,
I
a
’
and how he worked, what he did and vhet he failed to do. but there wds
a small area of difference. It did not tell us too much about job duties,
but it certainly told us hw the company valued job-parform=ca.. And
this vas the information that gave us the clues 8S to vhat tests to
select. Maybe thin point of view contains 8n idea we, should investigate.
Instead of looking at all the duties, functions, and rcsponsibflitiea of
the men, try to leern in what wrys we can spot a good one when we see one.
Let us not forget that there is a new set of tools at our disposal;
they are known as computers. Computers and their accessoF equipment
certainly ought to open nev d-ors to us. They have already opened many
doors. The biggest and most important is thet they permit us to do jobs
that were too big to do before. They handle much more data. Before
computers, how big a job of item analysis did you tackle? One hundred
high cases; one hundred low? That would be close to the limit of what
you would try to do. With comp::ters ve don’t think anything of using
revere1 thousand cases in our item analysis. We are tackling problem?
that are bigger erd bigger. Some of you may remember the day, it is before
my time I mfl;ht add, when six correlation coefficients would enable
one to e8rn a I% ;i. degree. As time vent on more correlations and more
complex correlat;.)n functions vere required. We are collecting and using
bigger and bigger quantities’of data in support of our efforts. These
are improvements, but not breekthrougha, because we have been using the
s8me old techniques, but with increased effectiveness, Computers do uuggest
that ve can do some things in ways never done before. For example: One
group of people for vhcxu I heve constructed tests .-or some yeers are so
bright I 8m not able to write items tough enough for them. These are the
science talent perticipants for the Westinghouee Scholare;.ips; these boys
and girls are really bright. Nevertheless, I’m still looking for talent
even in that group. I want a test vhich discriminates particularly among
those in the upper half of the contestants. I want a test which will be
reflected by a frequency distribution that has a positive skew. Some test
questions which come within the proper difficulty level may depend upon
remote and unimportant bits of information. Is it, for example, really
important to knov precisely hw many light years Arcturus is from the
Earth? Or even to know the answer within a thousand light years? This
fs the kfnd of question that is answered correctly by few people, but etfll
did not seem to be just what the situation demanded. Then ve tried multiple
choice questions; but instead of having just one right and 4 wrong answers,
we thought we would have one, two, three, four, or even five right ansvers.
A score of 1 is earned if all the alternatives are marked correctly, and
if there are ;qy errors the earned score is zero, This produced test
questions of appropriate difficulty and gave the right kind of frequency
.
.
27 i
-._ ._
,
.
, 3 t,
^>
.
. ,
I
i
1
I’
I
/