I__. - International Military Testing Association

In the second study, the response rates for the multiple
prob I em format interacted with administrations (i.e., days).
Rates for the multiple-problem format were always greater than
rates for the single-problem format; days 3 and 4 of the field
test they were approximately 10% greater (See Fig. 1). Practice
requirements, task definition, and task sensitivities were
comparable for the two different display formats. The I arger
response rates for the multiple-problem format are noteworthy
since faster rates are usually associated with tasks that have
superior psychometric propertles.
C 0
r
e
c t
P er
m in
DAYS
.-Q.. 33 Problems -+-- 1 Problem
Fig. 1: Response rates for an automated Number
Compar i son Task for a l-problem or a 33-prob I em
display. Each task was practiced four times previously
(5 min per administration).
I3 I SCUSS I ON
Automated NC was super ior to its paper-and-pencil
counterpart. The response rates, sensitivity to environmental
stressors, and test-retest reliabilities were greater for the
automated version than for the paper-and-pencil version. This
demonstrates that when performance tasks are automated, modified,
or developed they may have different psychometric properties than
thelr traditional counterparts. In this evaluation, the
automated verslon of the NC task possessed the best psychometr ic
properties.
Secondly, It is important that performance tasks be
evaluated during their adaptation or development. The success of
performance tasks is usually dependent upon their psychometric
properties (e.g. sensitivity. requirements for pract:;:; TtV$
test-retest reliabilities). Evaluation wi 11 ensure
psychometric characteristics of the task can be optimized and
that appropriate measures will be retained and used.
337