I__. - International Military Testing Association

In the fire fighting system prevention is realized by the number of units distributed over the
area before a fire is seen. False decisions mean forgetting to let the units search for and put out
fires by themselves.
Coordination is measured simply by the number of changing commands in the face of new
fires throughout the game.
Ability to decide
A high degree of decision-making ability means the capability to plan in a goal-directed manner
and to realize decisions quickly and precisely.
The following aspects are combined in the economic system: The time to control the system,
the frequency of postulated correct effects of decisions, and the rareness of decisions that do
not work in the system.
In the fire fighting system, the speed and accuracy of decision-making are rated in combination.
This means that the number of new fires that are dealt with in precise commands are summed
up and weighted by the average time lag between the time of the fire and the time that the
corresponding command is given.
4.3 Empirical results
4.3.1 Estimates of reliability
Concerning the economic system retests between different trials of system control do not seem
to be appropriate. Here we can expect content related changes in strategies that may influence
performance without being attributable to a lack of reliability. Empirical results from two
studies show stability of medium-level strategies, either accompanied or not accompanied by
stability in performance (see,Strohschneider, 1986; Funke, 1983).
In the fue fighting system, content-dependent changes in strategies are not to be expected. In
one experimental study (N = 50 university students) two versions of system parameters were
constructed which differed in the number and timing of new fires. The subjects had to control
each version for three trials in one session each. The correlations are lower between the first
set of trials than between the second set. Between the last two trials in the second version, all
correlations are higher than .80, referring to performance as well as to organizing and
decision-making ability. In a second study (N = 80 university students), one system version
had to be controlled for four trials. Performance data between the last two trials are correlated
.84, whereas organizing and decision-making ability is correlated .79 and .76, respectively.
These data on stability are accompanied by significant gains in performance as well as in
strategies from the first to the last trial. This is equally true for both studies.
4.3.2 Data on internal validity
As has been mentioned above, it should be tested whether the subjects are aiming at comparable
goals. If the goals are only vaguely defined, the subjects will probably define different specific
goals for themselves. Consequently, in our studies the subjects are given specific goal variables
. .