2003 IMTA Proceedings - International Military Testing Association

participated in informal class exercises to demonstrate the utility of the approach.
The afternoon session provided participants with hands-on practice using the 7-step
framework that had been presented in the morning session. Small groups of participants were
formed, with each group developing measures for a performance measurement objective of
interest. In the workshop invitation, participants were asked to bring their own objectives to
work on during the afternoon hands-on practice session. These objectives were used as the basis
for the afternoon session. Two facilitators from the research team were assigned to each group
to guide participants through the development of their measures.
WORKSHOP EVALUATION
Evaluation forms were developed to assess participant reactions to the morning briefing
and the afternoon hands-on practice. The morning evaluation form asked participants to rate the
performance measurement briefing on four criteria: Its ability to prepare them to (1) develop
good measurement objectives, (2) distinguish between outcomes and processes, (3) select an
appropriate measurement method, and (4) develop effective measures. Ratings were made via a
5-point Likert-type scale (5 = strongly agree, 1 = strongly disagree). Four open-ended questions
were then presented to assess whether participants’ expectations for the morning session had
been met, what they had found most useful about the briefing, what additional information
should be added to the briefing, and who could benefit from attending this type of briefing. A
similar evaluation form was developed for the afternoon session in that it required participants to
rate the hands-on practice on the same four criteria that were used to assess the morning briefing
and then presented two open-ended questions to assess what participants found most useful about
the hands-on practice session and how the afternoon session could be improved.
WORKSHOP 1
Morning Session Participants
Forty-four individuals attended the morning briefing on individual and team performance
measurement. Thirty-five of the morning participants (73%) completed the evaluation form. Of
those that completed the form, 11 were active-duty military personnel, 12 were civil service
employees, and 12 were contractors.
Reactions to Morning Session
Overall, participant ratings of the morning session were extremely positive. Eighty-six
percent of the attendees felt that the briefing had prepared them to develop good measurement
objectives. Seventy-nine percent reported that the morning helped them to distinguish outcomes
from processes. Eighty-six percent of participants indicated that the briefing had successfully
prepared them to select an appropriate measurement method and 80% reported that they felt
prepared to develop effective performance measures. These percentages are based on the
number of attendees who either agreed or strongly agreed with the objective statements.
689
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003