I__. - International Military Testing Association

Computer-based Assessment of Strategies in Dynamic Decision Making
1. Introduction
Wiebke Putz-Osterloh
University of Bayreuth
In psychological testing, computers are used primarily as economically efficient tools to
administer tests and to analyze and store individual data. This type of testing based on classical
tests is not the subject of this paper, however. Instead, I intend to speak about the uses of
computer programmes to simulate complex situations that call for dynamic decision making
(Kleinmuntz, 1985) or for complex problem solving as Diimer defines it (1978). In the
following, I will first discuss some reasons for complementing classical tests of intelligence
by other methods to extend the range of intellectual demands. Secondly, I will mention three
conditions that should be controlled if one intends to assess individual differences in complex
situations. Then I will summarize empirical results concerning individual differences in
problem solving strategies. Finally I will discuss some difficulties encountered in estimating
the external validity of strategies.
2. Reasons for expanding approaches to intelligence testing
Classical tests of intelligence (whether computer-based or conventional paper-and-pencil)
suffer from some common restrictions with respect to the intellectual demands they cover:
- Test items are static: Items have to be answered independent of the answers given previously
or to be given later.
- Test items are transparent and well defined: Individual differences in knowledge used and
strategies applied must be eliminated to make sure that only one single solution to each item
can be evaluated as the correct one.
- Intelligence is measured by the sum of the correct solutions to items that are to be solved as
quickly as possible: Time consuming processes such as the use of heuristic strategies are not
analyzable.
-Answers to test items have to be selected rather than constructed: Although in real-life
situations the rule is that one has to search for decision alternatives first and to select one of
these afterwards, such search processes are excluded from test-intelligence.
- Applicants often do not accept tests of intelligence as valid or fair predictors for personnel
selection. One approach to overcome the restrictions mentioned is to assess individual
behavior in multiple “real-life situations and exercises” as it is conceptualized by assessment
ten ter methods.
3. Conditions for the assessment of individual differences in decision making strategies
The following conditions are not met or even controlled when using assessment center methods:
- In complex situations there are multiple goals to be reached. Individual differences in decision
making depend on specific defined goals. If individual behavior is to be assessed, the goals
of each subject have to be controlled; otherwise the effectiveness ratings of individual
I....__ _ ,.
362