2003 IMTA Proceedings - International Military Testing Association

slightly modified for the second workshop. Specifically, we pre-selected an objective (i.e.,
Positively Identify Friendly Forces) from the Naval Mission Essential Task List to serve as the
basis for measurement development. The objective that we selected consisted of providing the
means, procedures, and equipment to positively identify friendly forces and distinguish them
from unknown, neutral, or enemy forces. This task included positively distinguishing friendly
from enemy forces through various methods that may include procedural, visual, electronic, and
acoustic, in addition to providing information to the force commander to aid in the identification
of unknown contacts. It was anticipated that groups of participants would develop performance
measures for the pre-selected objective at different levels of analysis (e.g., individual, team).
WORKSHOP 2
Morning Session Participants
Forty-nine participants attended the morning briefing on individual and team
performance measurement. Thirty-one participants (63%) completed an evaluation form. Of
those that completed the form, 21 were active-duty military personnel, 4 were civil service
employees, and 6 were contractors.
Reactions to Morning Session
Consistent with the first workshop, participants were asked to rate the morning briefing
against our four criteria using the 5-point Likert-type scale. Seventy-six percent of participants
felt that the briefing had prepared them to develop good measurement objectives. Sixty-six
percent felt that they were better able to distinguish outcomes from processes as a result of the
briefing. Sixty-nine percent of the respondents indicated that the morning had prepared them to
select an appropriate measurement method. Forty-three percent felt that the morning session had
prepared them to develop effective performance measures, but an equal number were neutral on
the subject. This result is most likely attributable to the fact that a number of participants
recognized that learning how to accomplish this task successfully would require practice.
In examining the responses to the four open-ended questions on the morning evaluation,
attendees felt that their expectations had been met in that “[the session] provided clear,
understandable methods for developing measures of performance” and “helped to develop a
calculated way to measure human performance.” In terms of the most beneficial part of the
morning session, participants cited the performance measurement steps, the background
information about human performance, and the examples and discussion. When asked how the
briefing could be improved, participants indicated that they would have liked more information
on the history of the Universal Task List, NMETLs, and how they drive training and readiness
assessments. Although there could have been greater discussion about the NMETL process, it
was assumed that participants already had some level of familiarity with NMETLs prior to
attending the current workshop. Finally, attendees reported that supervisors, individuals that
observe and assess performance, trainers, and anyone working with NMETLs could benefit from
attending this type of briefing.
691
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003