The role of metacognitive skills in learning to solve problems

Study III: added value of the task model 119
which suggested that it does not matter whether one follows a particular
strategy in order to choose interventions.
One drawback of the random simulation in the previous study is the
rather small random sample of event-intervention combinations. In KM
Quest, over 50 events exist and over 50 interventions are possible. And,
for each event a combination of various interventions can be chosen.
This produces a vast number of event-intervention combinations. A
simulation of only one experiment does not do justice to this vast space
of combinations of events and possible (sets of) interventions.
In order to create a larger sample of simulations the Monte Carlo
simulation technique was used in which a large number of simulations
of the experiment is produced. This technique enables sampling of large
multidimensional systems. The fixed sequence of events was taken as a
starting point, similar to the game set-up. The teams implemented an
average number of 5 interventions per event.
As students vary in the number of interventions chosen per event, the
simulations were made to match the variation in the number of interventions
chosen per event. This makes the simulation more sophisticated
than in the previous study in which this was not taken into account.
KMsim (De Hoog, Shostak, Purbojo, Anjewierden & Christoph, 2002;
Anjewierden, Shostak & De Hoog, 2002) was adapted in order to perform
the Monte Carlo simulations automatically. The average OEI score
after 100 simulations is 7.36 (SD = 0.78). A one sample t-test was used
in order to test whether the mean of OEI score in 100 simulations differs
significantly from the mean score of the student sample in this study
(8.27). The difference is significant (T = -11.64, p < 0.01); which means
that students perform better than the random simulations.
It appears that choosing appropriate interventions for particular events
yields a better game result than choosing interventions randomly. The
random sampling still produces a fairly adequate game result though,
which is similar to the default starting position of the game. Compared
to choosing no interventions at all (game is ‘played’ by the decay function
only) the random sampling still produces a very good game result
(see also chapter 4, figure 4.1).
Another question remains. Students in study II achieve an average
OEI score of 7.36 (SD = 0.56). In study III students achieve an average
OEI score of 8.27. Both samples are quite comparable on other variables.
Why are students in the current study performing better in the game?
Several explanations are possible.
One could assume that types of events vary in degree of difficulty. For
instance, non-KM related events and opportunity events could be more
difficult to choose interventions for than events that represent a threat