The role of metacognitive skills in learning to solve problems

74
4.3.4 ANALOG
Regarding the process data available in the log files, it turns out to be
far too time-consuming to analyse the log files by hand. Also analysis
with existing analysis software such as SPSS is difficult because of the extensiveness
and format of the data. Therefore, a tool was developed that
supports the analysis of behaviour in the game. This tool is called ANA-
LOG 3 (Christoph, Anjewierden, Sandberg & Wielinga, 2003). ANALOG
stands for ANAlysis of LOG files. Its aim is twofold: first it provides a
means to perform data analysis based on the log files generated by KM
Quest without losing information; second it supports protocol analysis.
The development of ANALOG occurred simultaneously with the process
data analyses that were conducted manually in this study. Several
requirements led the tool development. One requirement is that player
behaviour can be replayed both at an individual and at a team level.
Furthermore, the route through the game and interactions with team
members should be visualized in a simple way, such that interpretation
is quite straightforward. As for coding the protocols, one requirement
involves support for coding particular segments of communication. A
statistics module is required that enables export of data to Excel or
SPSS, supporting quantitative data analysis at various levels of detail
and of various selections of teams, players and quarters.
The main graphical user interface consists of several tabs that enable
browsing and coding of data, and that provide statistical and transitional
information (see figure 4.2).
Browsing The browse functionality is the most important functionality
to view data and to replay sessions. First, one is able to select and
filter the desired data. Selections involve choosing relevant teams,
players and quarters. The filter option provides the inclusion or exclusion
of specific types of activities (e.g. Opening a page, Editing a
page, Chat entries). Activities of players are visually represented in
a sequential manner and in terms of a simplified user interface of KM
Quest (see the center part in figure 4.2). The sequential view enables
‘walking through’ sessions of team members. A session consists of a
sequence of activities of one player starting from logging in to KM
Quest to logging out. Colours are used to distinguish different types
of locations, for instance, all steps that belong to the FOCUS phase
are blue. A location hierarchy is available (see figure 4.3) in order
to view all possible locations and their organisation. In the sequen-
3 ANALOG was implemented by A. Anjewierden, Human Computer Studies laboratory, University
of Amsterdam, The Netherlands