Technical b r Report - International Military Testing Association

generally t!lcse In which the tester Is comfort&bly superior
to the testee. 'I'hus the basic methodolop of personnel testing
and measurczent has developed around gaper-and-pencil Ins truments
to the rolnt where no one thinks It Inconkruous that
"Tests and 4casurements" textbooks devote upwards of 005 of
their space to paper-and-pencil tests. t
2. >!u!tInlIcltv of Yeasures. %'hen a osychometrlcian
----_
finally de?i,:e:; to racker .h.r:,nn rerformnnce'data on a slnulator,
he su:':‘crs an embarnsscent of riches -- scores or even
hundreds of !r.easures are available, all bossessinrz face
validity. ?I:-ure 2 shows data obtained hurlnc the lendlnrr of
a 707 slmul:ttcr; these items reyrcsent a selection from a
much larpzr n::aber of measures available.
3. L!1Ck of Parallel Items. in a landInz, there Is just
one Indicated alrsreeti at touchdown: to pet a parallel Item,
another lnn,llnr must 'ye conducted.
.
This lack of parallel
Items wIthIn :: riven crtneuq:er contrasts wit:? tke easv avallability
of p3rnllel Items for most paper-and-pencil test
domains.
4. I~I?:‘Icult~.@ ?n Comblnlnr Item Scores. With naner-
.-A.-. ---_____~
and-pencil tests, the ncmter of correctlv ansirered Items
usually s e rv c s 2 s an cverrlll or summary ne3scre; occasionally,
differential &Ir-htln- Is emnloved. 031:; rarely Is as coc?lex
an al~orlt~;:~ as successive hurdles used to derive a summary
measure from Item scores.
1
Nuck more connlel al~orlthns are reculred to
characterize Froficiency In tasks such as landlnr: a 707. For
that landin;: task, Items such ns Indicated airspeed, sink
rate, distance from runxzv threshold, distance from runway
centerline, r.r.c 3lrcr3ft attitude (Ditch! e keadlrz, and roll) --
all measure4 r;.t tcuch-33.n -- cannot L?e e aluated Independent
of each ot%** .L ., as Darer-and-nencll test btems are ceneraily
evaluated, Put r.:lst !'f cozSIned, usln;r 7 rather complex, yet
to be developed nl~orltt-.n.
5. In!?rn?IcabIlIt~- of Conventional Item Analvsis
Techniques?,tcz anaivs:s Is most >iItiqlly usec for muitiplechoice
Items. Kach multiple choice item renerally recresents
a separate kc!zr?vIcr snmnle, and the alternatives are discrete. -
When, as they often do,'ceasurements on a simulator cover
various nspccts of a sIn;?le behavior sample, rather than separate
behavlo? sazoles. and use continuous scales, rather than
discrete altcmntlves, conventional Item analysis Is lnappllcable.
6. p~~~v?Ila!~IlItv for :*!ensurement of Pe.h.avIors :!ot
Directly Hc!‘Ic.?tc:! In ‘.:?chlnc Innuts. in the simulator, the
����������� ��� ��������� � .~n--??cr. ?r: t'-e-* . . effect. the machine, are
;,xC?::rL' .; c t.-\ :'P-..':;:'C'.cI:t , :.1:t t!.e :':':,::':'.:e:- .- -_- :!*? * -..:!:ic.'. t?c-c
172
I
I
.