The role of metacognitive skills in learning to solve problems

Study I: Pilot 71
In conclusion, the aim of the HollandSky test was to measure low-road
transfer. There is some evidence that students are capable of applying
knowledge and skills acquired in KM Quest to a somewhat different case.
The effect is, however, rather limited. It is based on a selected group of
participants that completed all the items of the test.
Post-test results do not significantly relate to course grades. It appears
that the institutional course grade is based on different theories
of KM than KM Quest. A careful reviewing of the answers on the open
questions from the HollandSky test showed that students do not particularly
use the principles of KM Quest but rather the principles taught
in the institutional KM course. This is in fact negative transfer. Thus,
there is reason to believe that the institutional KM course and KM Quest
interfered. In a subsequent study it is therefore better that students first
play KM Quest before starting with the institutional course.
4.3.3 Metacognition
4.3.3.1 Retrospective measure: MSLQ
Cronbach’s alpha for the scale metacognition is 0.69. This is an indication
of moderate to sufficient reliability. On average, the subsample of
students scores 3.87 (SD = 0.57) on the scale metacognition. This shows
that students only marginally agree with using metacognitive skills in
KM Quest. No significant correlations exist between learning measures
and the self-reported use of metacognitive skills. A reason for this could
be the rather low number of cases in the subsample.
4.3.3.2 Development of the coding scheme
With regard to the selection of data on which protocol analysis was
performed, only those cases without missing data in the post-test were
taken into account (see page 68). Mean post-test scores of the teams
were calculated based upon the remaining team members. Members of
team 86 performed worst on the post-test (average score on the post-test
of the team member = 3.27, N = 3) and members of team 78 performed
best on the post-test (6.88, N = 2).
The objective of performing protocol analysis is to validate the coding
scheme. According to the theoretical model, communication is annotated
at the meta- and object-level. When applying the theoretical
model to the data in order to arrive at a coding scheme, two types of
deviations can occur. First, the protocols show categories of behaviour
not included in the theoretical model. Second, some of the proposed
categories in the theoretical model do not occur in the protocols (van
Someren et al., 1994). Three adaptations are proposed because these