Deliverable 4.4 - INSEAD CALT

AtGentive IST-4-027529-STP - Attentive Agents for Collaborative LearnersFeedback events (i.e. user clicks on a” smiley”)Students were able to click on a set of four “feedback” buttons in the form of “smileys”:• Happy• Neutral• Sad• ConfusedNot all feedback events were included in the analysis. Most importantly, clicks less than20 seconds apart are discounted (there are many instances where a student clicks manytimes over a few seconds; this is assumed not to represent a changing state of mind). 20seconds was chosen by examining the log files to be long enough to exclude multipleclicks. Also, clicks outside 8am – 3pm (scheduled classes) are discounted to removetests.The “Confused” smiley was removed from the Control group after week one (i.e. lessonson the 3rd and 4th of May), as the students expected it to give help in using the system(which it did not, and was not designed to do) and were unhappy that no help was given.Therefore, the analysis of “Confused” events only include such events from either groupthat occurred in week one.Results for the whole pilot show that the Control group clicked more often on “Neutral”,“Happy” and “Sad” feedbacks (see Figure 17), with both “Neutral” and “Sad” found to besignificantly different at the 5% level (see Table 6). Table 6 also shows the same test for“Confused” (means, CTL= 1.19, EXP= 2.36) – this was not found to be significant;however, while this contains data from both treatment groups, it only relates to the firstweek of the pilot (as explained above).54.543.532.521.510.50Neutral Sad HappyCTLEXPFigure 17 – Mean number of feedbacks for each student pair over whole of pilot, bytreatment groupDel 4.4: AtGentive Final Evaluation Report page 41