Abstracts - Earli

assessment of comprehension. Data are presented that indicate R-SAT exceeds the amount ofcomprehension variance explained in comparison to the G-M.Automatic coding of learners’ self-explanation when learning from diagramsShaaron Ainsworth, University of Nottingham, United KingdomRichard Forsyth, University of Nottingham, United KingdomDavid Clarke, University of Nottingham, United KingdomLaura Robertson, University of Nottingham, United KingdomClaire O’Malley, University of Nottingham, United KingdomTo understand learning with text and graphics, researchers typically take advantage of processmeasures such as verbal protocols. However, analysing an hour of protocols can take from ten tofifty hours. Therefore, we are interesting in exploring how techniques developed in computationallinguistics and machine learning could be used to help code verbalisations. Our approach(CODELEARNER) suggests that a system should have three key functions: accuracy (system andhuman coder assign the same code), economy (number of examples a researcher has to code totrain the system) and predictability (whether a system can estimate its own performance from asmaller subset of data). To test the system, we compared its performance to human coding of selfexplanationsgiven by learners studying concrete or abstract diagrams of the heart. There were23,330 words, sectioned into 1784 different segments and the human coder decided that 699 ofthese segments were self-explanations, 1022 were paraphrases and 63 were monitoring statements.CODELEARNER’s accuracy was 75% when trained with 1600 example segments. However, theCohen’s Kappa (0.52) would not be deemed satisfactory for inter-rater reliability in standardexperimental situations. CODELEARNER’s was more successful at economy exhibiting a steeplearning curves (see Figure 1), providing the system with only 300 coded segments allows it toachieve an accuracy rate of 70%. CODELEARNER was successful at predicting its level ofaccuracy with a larger training set. It can accurately predict how well it will do with datasets twicethe size as the one it was provided with.On-line methods to study dynamic representations processing: Eye tracking and comprehensionJean-Michel Boucheix, Universite de Bourgogne, FranceIn the current research about the comprehension of animated and multiple representations, on linemethods stay very few. But new eye tracking technologies allow to get more precise behaviouralindicators: fixations number and duration, transitions between selected area of interest in thepicture, precise eye trajectory, and scan paths. These measures can be newly combined with offlinecomprehension investigations. This presentation aims to show the relevance and also thelimits of such on-line methods to study multimedia comprehension. Two researches aboutdifferent kind of animated multimedia presentation will be exposed. The first study concerns thecomprehension of a complex mechanical system from an animated and controllable display. Thesecond research investigates, also with the eye tracking technique, the topic of the collaborativecomprehension in technical learning from multiple representations.Putting the assumptions to the test: Working memory processes in learning from text and picturesHuib Tabbers, Erasmus University Rotterdam, NetherlandsTheories stressing the involvement of working memory resources in learning from text andpictures are seldomly tested on their assumptions. Most of the times, design guidelines are testedon learning outcomes and not on their underlying cognitive processes. An interesting method for– 364 –