I__. - International Military Testing Association

PSYCHOMETRIC PROPERTIES OF A NUMBER COMPARISON TASK:
MEDIUM AND FORMAT EFFECTS
Banderet, L.E., Shukitt-Hale, B.L., Lieberman, H.R., Simpson’,
LTC R-L., Perez’, CPT P.J., U.S. Army Research Institute of
Environmental Medicine, Natick, MA, and ’ TEXCOM Armor and
Engineer Board, Advanced Technology Research Div., Fort Knox, KY.
ABSTRACT
Researchers adapting or developing performance tasks for
administration on personal computers are confronted with choices
that may affect the task’s measurement properties. To eva I uate
the effects of test medium, subjects completed Number Comparison
(NC) tasks administered with both paper-and-pencil and portable
computer media. Computer i zed NC proved super i or to
paper-and-pent i I NC ; Lhe automated version had greater completion
rates, ’ r-’ ‘Lbilities, and sensitivity to environmental
stressors”;~ypoxia and cold).
In a second study investigating task format, subjects were
tested with a computerized NC task which presented either 1 or 33
problems in each display window. Although the results were
similar for these two formats, the response rates for the two
formats were dependent upon the number of administrations. On
some of the later administrations, rates for the multiple-problem
format were 10% greater. Thus, formal evaluation during
adaptation or development of computerized performance tasks helps
ensure evolving tasks will possess reliability, sensitivity, and
other useful psychometric properties.
fNTRODUCTlON
Testing performance capabilities with tasks automated by
computers is more feasible today than ever before since computers
possess better displays, process information faster, execute
larger programs and data bases, store more information, and cost
less. When a performance task is adapted or deve I oped for
administration by computer, the subject’s output responses and
the instrument’s psychometric properties may change (Banderet et
al., 1989; Moreland, 1987).
We evaluated the automation of a performance task in two
studies.- In the first, an automated Number Comparison task (C-NC)
was compared to its paper-and-pencil equivalent (P-NC). In the
second study, the format of the display on the automated Version
Was evaluated. Displays with a single problem were compared with
displays with 33 problems. This report will describe the effects
Of task medium and format upon the psychometric properties Of an
automated NC task.
334