2003 IMTA Proceedings - International Military Testing Association

of situation in which Soldiers would use digital systems to execute tasks requiring collaboration.
The primary measures generated as the game is played are 1) the number of launcher that would
have been hit by each strike plan submitted and 2) the degree to which the two players on a team
chose to include the same grid squares in their independent strike plans. Only the first of these
will be discussed here.
Design. The primary independent variable for this experiment (ALL versus OWN) involved
training on the characteristic of the information-gathering assets used in the SCUDHunt game.
All participants received, as their first training module, an explanation of the characteristics of
the assets they would be controlling. Half of the pairs (the OWN condition) received a second
exposure to the same asset training; the other half (the ALL condition) received training in which
each participant learned the characteristics of the assets to be controlled by that participant's
partner. A secondary independent variable is the position to which participants were randomly
assigned, either "air" or "ground" controller. This position determined the particular set of
information gathering assets that were under the participant's control. The main dependent
variables in this experiment are 1) each participant's performance on the asset-knowledge quizzes
administered after each asset training module, 2) success at playing the SCUDHunt game, as
measured by the number of missile launcher locations correctly identified in a strike plan.
Procedure. Upon arrival at the laboratory, participants completed a preliminary questionnaire on
their experience with computers and computer games. The experimenter then explained that the
experiment would involve the participants playing such a computer game with a partner in
another room. First, they would watch a training video giving an overview on how the game is
played and explaining the concept of information-gathering "assets." They would then see a
video providing details on their assets, after which they would be asked a few questions about
what they had just learned.
Several computer-based training modules were then presented on 1) the overall aspects of
playing the SCUDHunt game and 2) the characteristics of the information-gathering assets used
in playing the game. Participants took paper and pencil quizzes on the material just presented
following each training module. Immediately after this training, the pair played a one-turn
practice game, to insure that the mechanics of playing the game were understood. After the
experimenters answered any question the participants might have, the pair played two complete
five-turn games of SCUDHunt. During these games, data were automatically collected on 1) the
messages participants sent to each other, 2) the degree to which grid squares chosen as targets in
the "strike plans" (submitted at the end of each turn) were identical for the two members of the
pair, and 3) the number of those chosen target squares that actually contained missile launchers.
Results and Discussion
The primary results are presented in Figures 1 and 2. Figure 1 depicts results for those
participants in the "air controller" position; it presents, for them, the main measure of success at
playing the SCUDHunt game – the number of SCUD launchers positions correctly identified –
on each of the five turns of both games played. Figure 2 presents the same data from the
145
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003