I__. - International Military Testing Association

lower) than those of soldiers who reported that they never supervise other soldiers. These results for
supervisory experience are slightly different than those obtained for supervisory training, where the
largest mean differences were found for the M-Effectiveness score. Perhaps this is because supervisory
experience sometimes involves making mistakes and leaming from the consequences of these mistakes
(i.e., learning to identify ineffective responses), but supervisory training is more likely to focus on the
identification of effective supervisory responses.
Table 4
Mean Situational Judgment Test Scores for Soldiers With Different Levels of
Supervisory Training and Experience
N M-Eff. L-Eff. M-L Eff.
Attendedone or more supervisory schools 560-603 4.97 3.50 1.47
-7
Attendedno supervisory school 327-371 4.81 3.62 1.20
How often required to supervise other soldiers:
Never 87199 4.87 3.63 1.23
Sometimes fill in for regular supervisor 294-327 4.86 3.58 1.29
Often fili in for regular supervisor 125-135 4.90 3.53 1.38
Regularly supervise other soldiers 391415 4.96 3.49 1.47
Conclusions
The results of the field test of the SJT indicate that this test is appropriately difficult for the target
sample. The five scoring procedures that were explored all resulted in scores with a reasonable amount
of variance among the soldiers in this sample. Internal consistency reliabilities were also quite high.
Based on all of the psychometric properties examined, the most promising score appears to be M-L Effectiveness,
which has an internal consistency reliability of .75.
The preliminary information obtained concerning the construct validity of the SJT provides evidence
that the SJT is a valid measure of supervisory job knowledge. The correlations of SJT scores with the
other job performance measures provide some support for the construct validity of the SJT. However, the
SJT also has moderate correlations with several measures of technical performance and with promotion
rate. Mean SJT scores for soldiers with different levels of supervisory experience and training indicate
that the knowledge or skill measured by the SJT is, to some extent, learned on the job and in supervisory
training.
REFERENCES
Motowidlo, S. J., Dunnette, M. D., & Carter, Cl. W. (in press). An alternative selection procedure: The
low fidelity simulation. Journal of Applied Psychology.
Smith, I. L. (1983). Use of written simulations in credentialing programs. Professional Practice of
Psychology, 4.2 l-50.
Tenopyr, M. L. (1969). The comparative validity of selected leadership scales relative to success in
production management. Personnel Psychology, 22,77-85.
273