The role of metacognitive skills in learning to solve problems

64
of learning on an individual level with respect to the institutional course.
Third, a ‘paper and pencil’ questionnaire (HollandSky) was developed
in order to individually measure low-road transfer as an indication that
meaningful learning occurred. This was done in a pre- and post-test.
This test involved the capability to apply knowledge and skills learned
in KM Quest to a slightly different case.
The questionnaire HollandSky was based on the learning goals of KM
Quest (see chapter 3) and consisted of two parts. The first part aimed at
measuring declarative knowledge and consisted of two open questions.
The second part aimed at measuring procedural knowledge. For this a
case was created with nine questions (3 closed and 6 open questions).
The case concerned a fictitious but realistic small travel agency called
HollandSky (Ediati, 2002). This company is comparable to Coltec but
differs with respect to the strategic aim. Coltec is a large product leadership
organisation, whereas HollandSky is a typical example of a small
customer intimacy organisation. Answering the questions correctly reflects
the ability to correctly transfer knowledge to a slightly different
case. Three events were presented in the case. The 3 questions per event
were about the type of the event, activities to employ, interventions to
choose and expected effects.
Two parallel versions of HollandSky were developed in order to prevent
test-retest effects. Additionally, having different but parallel question
items in pre- and post-test could yield higher motivation for filling
in the post-test, since items were new to students. The types of events
included in both versions were the same, the actual event was different.
For example, event 1 in version A of the test concerned the fact that the
general manager of HollandSky received complaints from a customer of
HollandSky with respect to the service it provides. An internal investigation
showed that the complaints began after the employee that usually
handles these contacts, could not do this anymore and her colleagues had
to take over (internal threat). The event in version B concerned the fact
that one employee is thinking of an early retirement (internal threat).
During the pre-test, versions were distributed randomly to players. During
the post-test, students received the version they had not seen before.
As some of the items concerned open questions, a standardised marking
model was constructed to support objective grading. Test outcomes were
graded on a ten point scale ranging from 1 (very bad) to 10 (excellent).
4.2.3.2 Metacognition
The use of metacognitive skills was measured retrospectively. After
task completion, students were asked to fill in an adapted version of
the MSLQ. Only the scale for metacognition was included. Students re-