Deliverable 4.4 - INSEAD CALT

AtGentive IST-4-027529-STP - Attentive Agents for Collaborative LearnersStudents had to assimilate a new working environment in addition to becomingproductive within that environment. With each lesson only 45 minutes in length, and onlyaround six lessons, that significantly reduced the time actually working on their learningtask.Conclusion: Consider the “overheads” – familiarisation with the software and the newworking environment and practices, and collection of experimental data – and allow timefor these in addition to the actual pilot.4.2.2 Results from PilotFirstly, we note that two possible confounding variables were identified. The first isAbility; the Experimental and Control groups were deliberately balanced for Ability(“Low”, ”Medium” or “Smart”) so this would not be expected to influence the main effects.The second is Gender: there were more girls in the Experimental group, and more boysin the control group (see Table 1). Both these variables are checked throughout theanalysis and where necessary their effects are described and discussed.Control n Experimental nMale 17 Male 11Female 9 Female 14Mixed 1 Mixed 3Table 1 - Distribution by gender for treatment groups (pairs)4.2.2.1. Assessment of students’ workEach student’s work was originally assessed by their individual teacher. While eachteacher was consistent in their marking, discussions with the teachers revealed thatthere was no overarching marking scheme within the school. It was therefore notpossible to compare students’ marks across classes. In order to allow this comparison,all the students were remarked by the AtGentive team (Barbora Parrakova and IngeMolenaar) at a meeting in Prague on 28-Jun-07 using an agreed marking scheme. Theremarking was “blind”, in that papers were taken at random, without reference to anystudent or teacher information. It was not considered useful to compare this remarkingwith the teachers’ original marks, since all the marking schemes involved are different.The items marked by the AtGentive team and considered reliable are listed in Table 2.Other items have been excluded due to large numbers of missing data points. Theresults are shown in Figure 14.Del 4.4: AtGentive Final Evaluation Report page 36