2003 IMTA Proceedings - International Military Testing Association

METHOD
PARTICIPANTS
The DEOCS had been administered to 522 participants at the time of the current
study. A random sample of 522 respondents to the MEOCS-EEO was selected for
comparison.
MATERIALS
Items had been taken for 14 scales from the MEOCS-EEO and revised for the
DEOCS: Sexual Harassment and Discrimination, Differential Command Behavior toward
Minorities and Women, Positive Equal Opportunity (EO) Behavior, Racist Behavior,
Religious Discrimination, Disability Discrimination, Age Discrimination, Commitment,
Trust in the Organization, Effectiveness, Work Group Cohesion, Leadership Cohesion,
Satisfaction, and General EO Climate. Two to five items from each scale were chosen
which previously research had shown to have good psychometric qualities (i.e., item-total
correlations, reliability, and discriminability).
PROCEDURE
A three-step process was followed in these analyses. First, Thissen’s (1991, 2003)
MULTILOG program was used below to obtain difficulty and discriminability
parameters (a and b’s) for the MEOCS-EEO and the DEOCS. Because these parameters
for the two versions were calculated separately, a common metric was needed. Then,
Baker’s (1995) EQUATE program was used to link the two versions. For each of the
scales presented below the parameters from the revised form of the MEOCS were
equated to those of the MEOCS-EEO. The transformation constants (A and K) are also
presented. Finally, following the transformation, DIF analyses were performed using
Raju, van der Linden, and Fleer’s (1995) DFIT program adapted for polytomous items
(Flowers, Oshima, & Raju, 1999; Raju, 2001) and Shealy and Stout’s (1993) SIBTEST
program adapted for polytomous items (Chang, Mazzeo, & Roussos, 1994).
RESULTS
A summary of the results can be seen in Table 1. Listed are the scales, number of
items in the scale, whether any items were reworded for the DEOCS, whether DIF was
detected by DFIT or SIBTEST. An examination of each of the scales is discussed below.
SIBTEST appears to be more sensitive to possible DIF than DFIT. Examination
of BRFs in this report suggests that the SIBTEST is overly sensitive. This also appears to
be true in previous research (Truhon, 2002). Chang et al. (1996) have reported that their
polytomous adaptation of SIBTEST is more likely to exhibit Type I error when there is
nonuniform DIF, which occurs with many items. This would help to explain the seeming
contradiction with Bolt’s (2002) finding that the SIBTEST had less power than DFIT.
Whether one uses the stricter criteria for DIF in DFIT or the looser criteria in
SIBTEST, it is noteworthy that there is a greater DIF in the reworded items than in the
items whose wording was left unchanged. These reworded items allow for a different
interpretation compared to the original version of the items by respondents. For example,
in the reworded version sexual harassment can involve women harassing men and racist
behavior can involve nonwhites discriminating against whites. Overall this suggests that
767
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003