I__. - International Military Testing Association

Table IV cont.
EM-46 13
Paygrade 1 N 1 N 1 % 1 N I N I % 1 N 1 N 1 %
E-5 43 16 37 42 9 21 25 6 24
E-6 44 18 41 42 22 52 31 12 39
E-7 11 5 45 8 7 88 4 2 50
E-8 8 E-9 1 0 0 1 1 100 2 2 100
TOTALS 99 39 37 93 39 42 62 22 35
Part Test Performance. In addition to evaluating any effects on total test
performance of randomizing the items it was also considered prudent to consider
any effects on domain performance. As indicated in Table V below, the results are
similar to those reported in Table Ill for total test performance. That is, the average
domain scores are quite consistent across test administrations with the 2-89
administration being somewhat easier for almost all domains across the three
administrations.
Table V
Average Domain Scores
BM-0110 EM-461 3
Randomized complete block design ANOVAs were computed for the domain scores
across the three administrations of each test and the results were not significant for
either the BM-0110 or EM-4613, (F[2,17] = 2.36) and (F[2,17] = .015) respectively.
Common Item Comparisons. Since it was not possible to use the same items in total
for each of the three test administrations, it was also necessary to evaluate the
effect, if any, on the subset of common items for each paired comparison. A twotailed
t-test was used to analyze the items common to each pair of administrations
and all results for both the BM-0110 and EM-4613 were nonsignificant at the .05
level, In addition, ANOVAs were calculated for each of the three administrations of
the BM-0110 and EM-4613 tests and the results failed to reveal any significant
differences at the .05 level of significance, (F[2,74] = .044) and (F[2,1461 = -720)
respectively.
Individual Item Statistics. The issue of any effect on item statistics of varying the
item’s position was investigated by comparing the item difficulty indexes (p-values)
of common items in each pair of test administrations as well as the item
277