I__. - International Military Testing Association

“Fzw%mmr-.-.‘~ .
which should be influenced in a specified direction. The objectively defined performance is
correlated with subjectively rated success after system control. In two studies all correlations
are highly significant: For the economic system the correlation is .52 (N = 100) and .48 (N =
48), and for the fire system it is .75 (N = 50) and .79 (N = 80).
An important characteristic of performance in system control refers to its partially ambiguous
meaning as performance level is not equivalent to a specific strategic variant. Following these
arguments, the internal validation of identified strategies should be the proof that these
differences are systematically related to performance level, whereas different strategic measures
shouId not be correlated too highly. In our studies there is clear evidence that differences
in strategies are systematically related to performance. Between studies there are differences
in the amount of common variance. In the economic system, decision-making and organizing
ability is correlated positively with performance, but this is not always significant (decisiveness:
.28 and .20; organizing ability .50 and .13; N = 48, N = 100). For fire fighting the
correlations are higher: decision-making ability with performance .58 (N = 50) and .45 (N =
80); organizing ability .58 (N = 50) and .63 (N = SO).
Further questions aim at the relation between the two strategic measures. In the economic
system, no systematic correlation between decision-making and organizing ability is found (N
= 48, N = loo), whereas in fire fighting there is either no relationship at all or no more than
9% common variance between the two measures (N = 80; N = 48). Finally, the generalizability
of strategies and performance between the two systems was also tested. In two independent
studies, one group of subjects first controlled the fire system and then the economic system,
while the other group worked the systems in the reverse order. A sequence effect is replicated
in the two studies: If the subjects control the fire system first and the economic system
afterwards, differences in performance as well as in organizing ability are correlated systematically
between the two systems (N = 25; N = 30), whereas no systematic correlations are
found if the systems are controlled in the reverse order. Despite some ambiguities in interpreting
the sequence effect, these data give evidence of the generalizability of strategies in system
control.
4.3.3 Dara on exrernal validity
If the systems do represent valid dynamic situations, experts in the simulated domain of reahty
should do’ better in controlling a system than novices do. As is to be expected, in two
independent studies (Putz-Osterloh, 1987; Putz-Osterloh & Lemme, 1987), university professors
(N = 7) and selected postgraduate students in management science (N’= 22) systematically
used more efficient strategies and achieved better performance scores in controlling the
economic system than unselected students (N = 29) did. For the latter subjects, the intelligence
test scores were controlled; they are not correlated with success in system control.
Following the logic of this expert-novice paradigm, in a further study, field-grade officers
(participants in a command and staff course) (N = 27) were compared with unselected students
(N = 30) in controlling the fire system.
Against expectations, no systematic differences between the two groups were found. DO these
negative results falsify a possible external validity of the system to predict success in higherlevel
military careers? There are two arguments that make me inclined to respond to this
question with a negative answer. First, the military subjects are not homogeneous with respect
. -. 366