zmWmQs

Designing Video for Massive Open Online-Education:
Conceptual Challenges from a Learner-Centered Perspective
Carmen Zahn, Karsten Krauskopf, Jonas Kiener, Friedrich W. Hesse
derable panels (see Figure 1), each of which contains the
extracted clip and a text field for annotation, comment or
other interpretation. Specific parts of the source video can
be extracted, which enables a user to direct the attention of
other users to what he or she is referring to. This process
has been termed ‘guided noticing’ (Pea, 2006). Each panel
with its comments constitutes a permanent external representation
of specific information within the dive, to which
users can resort whenever they decide to.
Figure 1. Screenshot of the online learning environment Web-
Diver TM (Pea et al., 2004).
The test materials consisted of a factual knowledge test
and a picture recognition test. The pre-test and post-test
of participants’ factual knowledge of the historical context
were created from information taken from secondary school
history textbooks. These tests were given in a multiple choice
format, where either one (pre-test) or multiple options (posttest)
per item were correct. The picture test consisted of 28
pictures, half of which were scenes taken from the original
newsreel and half of which were distractors from a different
newsreel (same genre and period).
Measures. To assess learning outcomes in terms of general
content knowledge acquisition, we administered the factual
knowledge tests (pre- and post-test) and the picture recognition
test after the collaboration phase. Total test scores
were computed, resulting in a theoretical maximum of 12
points in the pre-test and 45 in the post-test. To assess task
performance, the participants’ contributions from the saved
panels (WebDIVER protocols, see below), including their
selections from the video, annotations and comments, were
analyzed. Precisely, our analyses were based on the overall
number of the created panels, the number of panels, where
details were selected by using the WebDIVER’s selection
frame, and the number of comments including their length
in words. Additionally, we analysed the quality of the dyads’
comments by coding (a) aspects of contents covered
in relation to the learning goal and (b) aspects of collaboration
quality. For coding of the comments, we developed
two coding schemes. The first one – coding scheme I (see
below) – was developed to assess the quality of the panel
comments. The second one – coding scheme II (see below)
– was developed to assess the overall quality of interactions
within dyads. Here, screen videos were viewed in addition to
examination of comments to determine which comment was
written by which collaboration partner thereby counting
panels created in partnership by both participants of the
dyads together, and for categorizing different kinds of social
interaction in the comments. All comments were coded by
two observers.
Coding schemes. Coding scheme I for the quality of the
comments consisted of the following categories: utterances
addressing historical content of the newsreel, utterances
addressing filmic style of the newsreel, and utterances integrating
aspects of historical content and filmic style, respectively.
Units for the utterances were defined as sentences or
sentence fragments. On the basis of these categories, Diver
protocols were coded by two independent, trained raters.
Interrater-reliability ranged between Cronbach’s α = .80
and .98. Coding scheme II rating the comments exchanged
within dyads was developed in two steps: First, two observers
analyzed and discussed the WebDiverTM protocols
and considered relevant literature (e.g., Stahl, Koschmann &
Suthers, 2006) in order to derive indicators for establishing
different categories of collaboration, including coordinating
and communicating activities. Second, a collaboration index
was calculated. The categories applied in analysis were: 1)
double references as an indicator of collaboration in general;
2) proposals for work structuring as an indicator of
coordination activities; and 3) referencing one partner’s
utterances or directly addressing the other partner as an
indicator for communication. The coding results were then
integrated by weighing the number of utterances of category
1) by factor three, because they were considered the
strongest indicator of collaboration. The result was then
added together with the number of utterances in categories
2) and 3) to form the collaboration index. This collaboration
index and the number of panels created in partnership were
used for further analyses. Again, two independent raters
performed the analysis. Interrater-reliability ranged from
Cronbach’s α = .92 and 1.0.
Data analyses. For data analyses, we grouped the dependent
variables into three levels (see Table 1): First, a cognitive
level with regard to knowledge acquisition, ensuring effectiveness
of online-learning in the two conditions (factual
knowledge and picture recognition performance). Second, a
surface level of effects on collaboration and learning, where
we compared the two conditions with respect to the variables
describing overall collaborative activities (number of
comments, length of comments, number of panels created
in partnership, and collaboration index). Third, pointing at
deeper level effect on collaboration and learning we looked
at quantitative and qualitative indicators for more knowledge
intensive collaborative activities (panels referring to
details, and utterances addressing either historical content
or filmic style of the newsreel and utterances integrating
aspects of historical content and filmic style).
Research Track |163