2003 IMTA Proceedings - International Military Testing Association

There is a link between test construction and test equating. This is illustrated in a
statement by Mislevy (1992),
Test construction and equating are inseparable. When they are applied in concert,
equated scores from parallel test forms provide virtually exchangeable evidence about
students’ behavior on the same general domain of tasks, under the same specified
standardized conditions. When equating works, it is because of the way the tests are
constructed…(italics in original, p. 37; from Kolen & Brennan, 1995, p. 246).
Dodd, De Ayala, & Koch (1995) have described a procedure to use CRFs to
calculate item information functions (IIFs). These IIFs can be added together to produce
test information functions (TIFs). Wu (2000) developed a model management system to
develop comparable tests making use of TIFs. These equating procedures can then be
used to develop comparable forms of the DEOCS. In this way IRT can be used to
develop alternate forms of the MEOCS and DEOCS (see Cortina, 2001) rather than
compare forms after the fact, as was done in this study.
The kinds of analyses described above set future directions for the DEOCS. They
are frequently used in computerized adaptive testing (CAT). Most CAT has been done in
the area of ability testing. However, CAT has been successfully applied to attitude testing
(Koch & Dodd, 1990). A CAT-version of the DEOCS could establish a person’s response
level on the different scales in the DEOCS with a minimal number of items.
Finally, the DEOCS has begun to go on-line. DIF analyses should be used to
compare on-line responses to paper-and-pencil responses to the DEOCS. While previous
DIF research suggests that administration format does not make a difference for attitude
scales (Donovan, Drasgow, & Probst, 2000) and evaluations (Penny, 2003), it would
useful to verify this.
REFERENCES
Baker, F. B. (1995). EQUATE 2.1: A computer program for equating two metrics in item
response theory [Computer program]. Madison: University of Wisconsin,
Laboratory of Experimental Design.
Bolt, D. M. (2002). A Monte Carlo comparison of parametric and nonparametric
polytomous DIF detection methods. Applied Measurement in Education, 15, 113-
141.
Chang, H., Mazzeo, J., & Roussos, L. (1996). Detecting DIF for polytomously scored
items: An adaptation of the SIBTEST procedure. Journal of Educational
Measurement, 33, 333-353.
Cortina, L. M. (2001). Assessing sexual harassment among Latinas: Development of an
instrument. Cultural Diversity and Ethnic Minority Psychology, 7, 164-181.
Dodd, B. G., De Ayala, R. J., & Koch, W. R. (1995). Computerized adaptive testing with
polytomous items. Applied Psychological Measurement, 19, 5-22.
769
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003