I__. - International Military Testing Association

Figure 2 shows temporal effects at auto mechanic test sites.
Site A experienced a slight drop in agreement in the middle time
period. The decline in reliability was noted at the time, so
counselling and retraining were conducted, resulting in an
increase during the final period. Site B agreement increased
during all three periods. The overall increase in agreement at
both sites is natural given increasing familiarity of the TAs
with the scoring standards over time. The fact that the increase
is relatively small indicates that the initial training program
prepared the TAS very well. Again, this points to test effects
as a likely cause of the differences in reliabilities.
This same trend carries over into the helicopter mechanic
t reliabilities despite some differences in their HOPTs. Whereas
all automotive mechanics were given the same HOPT, each
helicopter mechanic MOS had its own test. Helicopter MOSS were
tested sequentially, so temporal effects are evident across
aircraft type, as shown in Figure 3. Since test order varied
across site, increased reliability is not indicated left to right
for both sites. Test order for Site C was CH-53A/D, CH-46,
CH-53E, and UH/AH-1. Site D order was UH/AH-1, CH-46, and CH-
53E. No CH-53A/D were tested at Site D. Taking this test order
into account we see that agreement increased over time at both
sites, except for the CH-46 at Site C.
Agasmcnt between +-7wy md shodor ted odnhistrotorr b y oicroft
Fig. 3: Agreement by Aircraft Fig. 4: Agreement by Interview
Ratings
The drop in CH-46 agreement is explainable in terms of
variation in test conditions. At Site D, all examinees in 2
particular MOS were tested on the same aircraft. At Site C, each
unit set up its own aircraft, resulting in changing conditi