2003 IMTA Proceedings - International Military Testing Association

the operational environment and, of both the immediate and long-range goals are what will
ensure maximal SME effort.
C. Participants should be advised that small-group processes will be used and that
group dynamics training will be the first step in the process. Materials on "groupthink",
group forming etc. should also be included in this package. At the first session, some group
exercises should be conducted to demonstrate what the materials have described.
D. Finally, the read-ahead package should include a homework assignment.
Participants should be instructed to read all of the read-ahead package and then begin
thinking and working on the first objective. A day-by-day agenda should be provided as
well.
NOTE: Along with the read-ahead, the facilitator must prepare a complete set of protocols
for the participants to be given to them when they arrive. Besides having the read -ahead
materials, the protocols also have the group processes materials and a step-by-step breakout
of the objectives, sequencing and, how each objective will be carried out. The protocols are
the structure to keep the groups on the process track.
THE SEQUENTIAL PROCESS
The optimum size of the workshop is 10 persons broken into 2 sub-groups of 5. Upon
arrival each SMEs should be provided with a final agenda and the protocols that specify how
each objective will be accomplished. The next step is to proceed with instructions and
exercises in group dynamics and consensus. Work on the objectives can now begin. An
iterative step-wise process should be used where each objective is accomplished by
anonymous and individual means. Sub-group discussions are then held to achieve consensus.
Finally, the intact (the 2 sub groups meet together) for a group discussion to achieve final
consensus. The iterative, step-wise process it to go from a 5 person group meeting on subobjectives
to an intact 10 person meeting to make a group decision on the main objective.
DISCUSSION
One obvious candidate for the SGDP methodology is Operational Testing (OT). The
goal of any OT is to determine the system’s capability to perform its mission in the
operational setting and to determine and evaluate core operational effectiveness and
suitability problems. The SGDP can be modified so that SMEs can use it in OT.
A small set of carefully selected SMEs would be used to face-validate the existing scenarios.
This SME set also would ensure that all the necessary operational issues were embedded in
the scenarios. Dumas and Redish state, “a good scenario is short, composed in the user’s
words (not the products), and is clear so all participants will understand it”. A second group
of SMEs would also assist in the test sequencing as well as the techniques/scalings to be used
in workload analysis. While the process of multiple SGDP workshops may seem lengthy, it
is not the case. The Workshops would run consecutively, each new one beginning with the
data from the prior one---and each SGDP workshop only would need approximately 4 days to
complete. The experience and insight as to the realities of the operational arena of the SMEs
are both needed and invaluable. Upon completion of the actual OT, all SME members of the
2 previous SGDPs would be convened to interpret the results and to make a set of
recommendations. These recommendations may include retrofit, modification, and training.
The two SGDP groups would work independently at first. Then, as they finished, they would
convene into one group to finalize their results.
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003
3