2003 IMTA Proceedings - International Military Testing Association

METHOD
Participants
Participants were ten experts with extensive experience in human performance
measurement and training. Many also had experience working for the Navy or other
military branches. All had PhDs in various areas of Psychology, with the majority having
a Ph.D. in Industrial/Organizational Psychology.
Participants’ collective experience in the human performance measurement arena
included test development and assessment, performance model development, job
analysis, and performance appraisal measure development. Their collective training
experience included developing training evaluation measures, using assessment centers
for development, developing training programs, and facilitating training. In addition,
several had evaluated training programs and developed competency models for the Navy.
Measures
The four rating formats included checklist, frequency, distance and discrepancy,
and rating scale. Participants were given definitions and examples of processes and
outcomes and told to rank their first, second, and third choice of format for measuring
each process and outcome (Interview guide is available from the author.) If they felt
other formats were necessary, they were instructed to add them and explain their reasons
for doing so. In addition, participants explained their rationale for choosing their first
choices for each process and outcome. Finally, they provided demographic information
on themselves.
663
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003