Deliverable 4.4 - INSEAD CALT

More documents

Recommendations

Info

AtGentive IST-4-027529-STP - Attentive Agents for Collaborative Learnersinformation. For the remaining student pairs, the shape of the transition graph is more orless random. This means that about half of the student pairs used the systemapproximately in a same style, but the other half did not have a recognizable pattern ofusage.One important idea that AtGentSchool inherited from the Ontdeknet system is theinteraction with expert. Students have a chance to pose questions to a domain expertwho will give a personal reply. It can be seen from the state transition visualisations thatthere were only six student pairs who never took advantage of this feature.The flow visualisations capture the events of the pilot for a student pair into a single view.If these visualisations are inspected as overall patterns then three shapes emerge: a lotof activity at the beginning and steady decline towards the end of the pilot, approximatelythe same amount of activity throughout the pilot, and steady increase of activity towardsthe end of the pilot. The steady activity throughout the pilot is the most common pattern(29 out of 55 student pairs) and represents perhaps the desired state as the closerinspection of event flows reveal. These students are following the path that theAtGentSchool system is supposed to guide them through. The other patterns (decliningactivity, 18 student pairs, and growing activity, 8 student pairs) seem to signal some kindof problems, perhaps related to the schedule pressures that were reported. Some of theclasses had to carry out the pilot in a relatively short period of time.Finally, the log visualisations provide an alternate view to the task times because of theslightly higher abstraction level. Overall, according to the visualisations, the studentsspent most of their time preparing the goal of their work, the paper about living in abroad.Almost as much of their time was spent on the discussion forum where they had achance to exchange opinions about their projects. Surprisingly large portion of their time– almost as much as was spent on the forum – was dedicated for maintaining theirpersonal profile on the system. The students in experimental and control groups spentroughly an equal amount of time on the paper and diary tasks, but there was a cleardifference on the time spent in discussion forum. The control group spent only one thirdof the experimental group’s time in the discussion forum.4.3 Pilot study - AtGentNetData collected as part of the AtGentNet pilot, and available for analysis, is as follows:• log files• questionnaire responseso students feedback• post-trial interviews4.3.1 Results from Pilot4.3.1.1. Log filesThe log files used for the evaluation are text-format files (the platform has the ability toexport the activities in various formats, such as text or Atom) with one line for each“event” (such as the user amending their profile, or reading a posting). There is one fileper student. However, of the 27 participants, one student from each group did not usethe ICDT platform, giving data from 12 students in the Experimental group and 13students in the Control group.Del <strong>4.4</strong>: AtGentive Final Evaluation Report page 56
AtGentive IST-4-027529-STP - Attentive Agents for Collaborative LearnersIt must be noted that this is a small quantity of data. A further problem is a lack ofhomogeneity within the data; often a small number of participants have used a functionfar more than others. However, the others tend to be disparate, making the removal ofdata as “outliers” highly problematic, since removal of one outlier usually invites the nextvalue in turn to be considered in similar terms. The decision was therefore taken to usethe data as obtained.Logged events comprise Date, Time, Event type, Event details. The main eventsavailable are:• User changes of screen• Creating / Editing / Reading postings• SearchesA typical log file would contain data of the form:17/09/2007 09:40:34 visiting place 'forum exchanges'17/09/2007 09:40:37 visiting place 'forum exchanges_exchanges'17/09/2007 09:40:48 reading message 'exchange on coaching'OBU created an analysis program that reads in all the log data and picks out selectedevents, outputting totals for further analysis.Number of logins per monthFirstly, the number of times participants logged in each month was calculated. This givesa broad view of how the system was used. Figure 19 shows logins for the months May toOctober, by treatment group (Control and Experimental) and as a total.As described in Section 2.2.8, the pilot began with an introduction to AtGentNet for TRIMparticipants at meeting in Lidköping, Sweden on 24th May 2007. However, many of theparticipants were sent details of the platform prior to this date, allowing them to log infrom mid-May onwards.On the 4th of September 2007 participants were sent emails inviting them to take part inthe Eagle Racing simulation (see Section 2.2.7). Therefore logins for September includeany specifically for this. (The racing simulation was over by 10th October, but adiscussion forum continued throughout October.)20181614121086420May Jun Jul Aug Sep OctCTLXPTOTALDel <strong>4.4</strong>: AtGentive Final Evaluation Report page 57
Page 1 and 2:
ATGENTIVEIST-4-027529-STPDELIVERABL
Page 3 and 4:
AtGentive IST-4-027529-STP - Attent
Page 5 and 6:
Page 7 and 8:
Page 9 and 10:
Page 11 and 12: AtGentive IST-4-027529-STP - Attent
Page 13: AtGentive IST-4-027529-STP - Attent
Page 22 and 23: May 2007AtGentive IST-4-027529-STP
Page 60 and 61: CollaborationAtGentive IST-4-027529
Page 88 and 89: DescriptionAtGentive IST-4-027529-S
Page 90 and 91: Integratability:AtGentive IST-4-027
Page 112 and 113:
Page 114 and 115:
Page 116 and 117:
Page 118 and 119:
Page 120 and 121:
Page 122 and 123:
Page 124 and 125:
Page 126 and 127:
Page 128 and 129:
Page 130 and 131:
Page 132 and 133:
Page 134 and 135:
Page 136 and 137:
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Page 154 and 155:
Page 156 and 157:
Page 158 and 159:
Page 160 and 161:
Page 162 and 163:
Page 164 and 165:
Page 166 and 167:
Page 168 and 169:
Page 170 and 171:
Page 172 and 173:
The effects of a computer agent’s
Page 174 and 175:
Page 176 and 177:
Page 178 and 179:
Page 180 and 181:
4. ResultsAtGentive IST-4-027529-ST
Page 182 and 183:
Page 184 and 185:
Additional pedagogical investigatio
Page 186 and 187:
Page 188 and 189:
Page 190 and 191:
Page 192 and 193:
Page 194 and 195:
Page 196 and 197:
Page 198 and 199:
Page 200 and 201:
Page 202 and 203:
Page 204 and 205:
Page 206 and 207:
Page 208:
show all

Deliverable 4.4 - INSEAD CALT

Create successful ePaper yourself

Delete template?

Save as template?