I__. - International Military Testing Association

Test Mode Supervisory
WRITTEN TESTS Situational Judgment Test
(Mean of effectiveness
weight for "M' responses
minus effectiveness weight
for "L" responses)
-
Job Components
CommxT MOS-Specific
Job Knowledge Tests Job Knowledge Tests
of Comnon Tasks of MOS-Specific Tasks
(Percent items correct) (Percent items correct)
JOB SAMPLE TESTS Supervisory Role-Plays Hands-On Tests Hands-On Tests
(Mean across role-plays of Cornnon Tasks of MOS-Specific Tasks
of ratings on 3-point (Mean across tasks of (Mean across tasks of
effective behavior scales) percent steps passed) percent steps passed)
RATINGS Rating Scales - Army Wide Rating Scales - Army Wide Rating Scales -
Supervisory Dimensions Non-Supervisory Dimensions MOS-Specific Scales
(Mean across dimensions (Mean across dimensions (Mean across dimensions
of supervisor ratings on of supervisor ratings on of supervisor ratings on
7-point rating scales) 7-point rating scales) 7-point rating scales)
Figure 1. Testing instruments providing coverage of each job component, by
test mode.
Table 2
Statistical Characteristics of Test Instruments Across Nine MOS
Supervisory Comnon MOS-Specific
Mean SD Rel. Mean SD Rel. Mean SD Rel.
Situational Judgment Tests 1.37 .60 .75
Job Knowledge Tests 65.4 12.5 .79 64.9 13.5 .73
Supervisory Role-Plays 2.26 .42 .71
Hands-On Tests 72.6 15.4 .46 69.4 19.5 .44
Rating Scales - Army-Wide 4.49 1.06 .50 5.13 1.13 .48
Rating Scales - MOS-Specific 5.19 0.97 .43
Note. Situational judgment test results ranged from -.77 to 2.57 (thus the mean score of 1.37 is roughly
equivalent to a score of 4.46 on a 7-point scale, with a standard deviation of 1.26): reliability estimate
is split-half on items, corrected to test length.
Job knowledge tests and hands-on test scores are proportions correct: reliability estimate for job knowledge
tests is the median across MOS of a split-half on odd-even items, corrected to test length; the reliability
estimate for hands-on tests is the median across MOS of the split-half on task scores, corrected to number
of tasks.
Ratings were made on a 7-point scale, where a 1 represents poor performance; reliability estimates are onerater
reliabilities across dimensions, using the median across MOS for MOS-specific ratings.
Role-play ratings were made on a 3-point scale, where a 1 represents less effective supervision; reliability
eStim&eS are the median one-rater reliability across items, averaged across the three role-plays.
544