The role of metacognitive skills in learning to solve problems

88
opportunity events (instead of threats), or events that have a direct effect
on the market share of Coltec. These were not taken into account in the
technical perspective because interventions do not directly counteract
their effects.
For the conceptually correct interventions, the game developers (experts
in KM) created a list that at a conceptual level prescribed which
interventions belong to which events. Again, for non-KM events, no
interventions were prescribed. For opportunity events, only a recommendation
of interventions was given.
Unfortunately, the two lists do not overlap a great deal. For specific
events less than 10% of the interventions are the same.
5.2.3.2 Learning results
The test that measures meaningful knowledge is called KMQUESTions.
KMQUESTions is an electronic test tool that consists of a number
of multiple-choice items. The items were explicitly based on the learning
goals for the acquisition of declarative and procedural knowledge formulated
for KM Quest. KMQUESTions consists of two parts, analogous
to the two types of knowledge required. For each learning goal an even
number of items was developed. The even number of items served the
purpose of creating two parallel versions of the test in order to prevent
test-retest effects. The number of items belonging to a particular learning
goal varied. A correctly answered item resulted in a score of 1. Items
that were not correctly answered received a score of 0. For each learning
goal a score is calculated weighted by the number of items that belong
to that learning goal. The final score on the test reflects the average
score on the learning goals. Each learning goal was assigned the same
weight. In total, 102 items were developed, the parallel versions each
containing 51 items. All items had to be answered in the context of
KM Quest. The final score (proportion of correctly answered items) on
the post-test of KMQUESTions was the grade students received for this
part of the institutional course on KM. In figure 5.1 an example of an
item of KMQUESTions is given.
5.2.3.3 Metacognition
The scale metacognition of the MSLQ was used in order to measure
the self-reported use of metacognitive skills of students. Question items
were administered to students prior to task performance, in order to
investigate how students use these skills in general. This represents a
trait measure instead of the state measure as was used in the pilot study
in chapter 4. A trait is a personal disposition that is relatively stable over
time. A state is a relatively unstable characteristic that is transitory over