I__. - International Military Testing Association

-- .- ..-- -_*- --.._ --l_*y -r
The M-Correct and L-Correct scores have less desirable psychometric properties than the scores
obtained using the other three scoring procedures. In addition, these two scores contain information that
is very similar to the information provided by the M-Effectiveness and L-Effectiveness scores respectively,
because they are based on the same sets of responses. Thus, results reported for the remainder of the
analyses will not include these two scores.
Preliminarv Information Concerninp Construct Validity
Table 3 shows the correlations of the three remaining SJT scores with scores from the other job
performance measums. The SJT SCOIW cormlate moderately with a composite of scores on the three
superviscry simulations. The SJT scores also have moderate correlations with the performance rating
composite called Leading/Supervising. Correlations with the other performance rating composites are
slightly lower. Correlations with scores on the job knowledge tests are quite high, but this is not surprising
in view of the fact that these are also paper-and-pencil tests. Finally, the SJT scores have moderate
correlations with a variable called “grade deviation score”, which is essentially promotion rate. Promotion
rate might be seen as an overall measure of success as a soldier.
Table 3
Correlations Between SJT Scores and Other Job Perfomlance Measures
Performance Rating Composites3
Effort/ Grade* Supervisory
Leading/ Technical Personal Military Job Deviation Simulation
Supervising Performance Discipline %uing Knowledge’ Score Composite4
M-Eff. .24 .21 .20 .ll .40 .20 .20
L-Eff. ~18 -.17 -.15 -.06 -.34 -.20 -.I6
M-L Eff. .22 .21 .18 .lO .40 .22 .20
___- -....-. ___.--
1 Weighted mean across nine MOS; sample size per MOS ranges from 38 to 146.
2 This variable is essentially promotion rate; sample sizes range from 849 to 919.
3 Based on pooled peer and supervisor ratings. Sample sizes range fran 855 to 907: a con-elation of .07 is siwifiant at the .05 level.
4 Composite of scores ftwft three. simulations: personal counseling, disciplinary counseling, and training. Sample
aizcs range from 873 to 909, a correlation of .07 is significant at the .OS level.
Table 4 shows the mean SJT scores of soldiers who reported various levels of supervisory training.
Soldiers who had attended no supervisory school at all scored almost a half a standard deviation lower
than those who had attended one or mom supervisory schools. One potential confound in this compati.son
is that the opportunity to attend supervisory schools varies, and decisions concerning which soldiers
are given the opportunity to attend these schools may be influenced by their effectiveness as soldiers or
as supervisors. As a result, it is possible that these mean SJT score differences were obtained because the
more effective soldiers were given the opportunity to attend supervisory training. However, regardless of
whether these differences are the result of differential opporturities or training in the relevant supervisory
skills, these mean score differences provide some support for the construct validity of the SJT as a measure
of supervisory skill.
Mean SJT scores are also reported on Table 4 for subgroups of soldiers identified by how frequently
they reported supervising other soldiers. For all three SJT scoring procedures the expected pattern was
found; soldiers who reported that they supervised other soldiers more frequently obtained better SJT
sCOms. The largest difference is for the L-Effectiveness score. Soldiers who reported that they regularly
supervise other soldiers obtained L-Effectiveness scores almost half a standard deviation better (i.e.,
272
I