I__. - International Military Testing Association

c
mechanic among the TAs (the rest were from the other three
aircraft). The reduced agreement may partly reflect this
diminished commonality of experience among the TAs. Even so,
over time, the continuing training program and skill transfer
across aircraft resulted in overall increased reliability.
Figure 4 plots agreement versus the initial TA interview
ratings. The strongest correlation with agreement was for TA
applicants who rated high in test administration, public
speaking, and administrative experience. The negative effect of
maintenance and mechanical familiarity may indicate a bias
resulting from experience. Yet in all cases, reliabilities were
acceptable. Interestingly, among all MM TAs, there was no
significant difference in between TAs who had also senred on the
infantry project several years earlier and the mechanics hired
for this project.
Conclusion
The high reliabilities found in the preliminary analysis are
encouraging. They indicate that the TA training program was
sound, scoring was well standardized across sites, and that the
HOPT steps were discrete, and consistently measurable. There
were also no indicators. of any significant test effects or other
systematic problems with the test that would preclude achieving
the high validities obtained in the infantry study. Finally, the
results have implications for HOPT Test Administrator selection.
This analysis seems to indicate that such qualities as previous
test administration experience and public speaking are more
important than experience in the particular field being tested.
Reference
Maier, M. ,H. (1988). On the Need for Quality Control in
Validation Research. Personnel Psvcholosv, 41, 497-502.
540