Deliverable 4.4 - INSEAD CALT

AtGentive IST-4-027529-STP - Attentive Agents for Collaborative LearnersThere is no significant difference between the Experimental and Control groups in theirplatform use during this time. This suggests that the AtGentive enhancements were notfound to help the Experimental group with collaboration.Taken together then, pre-Eagle Racing, students were busy and focussed on the ITMcourse. The AtGentive enhancements appear to have helped with performance but notcollaboration. During Eagle Racing, the reverse was seen, with AtGentive enhancementsappearing to have helped with collaboration but not performance.This suggests that performance is supported by the AtGentive enhancements where thestudents’ need is to obtain information, whereas collaboration between students issupported by the AtGentive enhancements where the students’ need is to work on jointprojects. Support for collaboration with lecturers does not appear to be affected by theAtGentive enhancements.A meta-level factor at work here appears to be that of motivation – participants weresuccessfully supported by the AtGentive enhancements according to their motivation.When intending to acquire information, the system supported their performance in thistask. When intending to collaborate with other students, the system supported theircollaboration. Thus, the AtGentive enhancements supported the intended activity, ratherthan promoting new activity.It should be noted that the Experimental group scored higher on their self-assessedunderstanding of the platform (all agreeing with “I fully understand how to use the ICDTplatform”, rather than the Control group, for whom all-but-one disagreed). Similarly, theExperimental group scored higher on “I understand all the headings on the ICDTplatform”. Also, the Experimental group scored higher with “At school I always receivedgood grades”.Generally, the interviews revealed that participants found the platform difficult to use. It ispossible that some of the Experimental group’s greater participation and collaborationmay be due to a better understanding of the platform. If this were the case, bettersupport for beginners may be indicated in future systems that support attention usingperception. Interviewees reported that there was “too much information”. Support forbeginners could involve introducing features over time, as they get used to the system asa whole.AttentionThe participants’ use if the ICDT system was uncontrolled from the evaluationperspective. Participants could use any computer at any time, may have a slow orunreliable connection, may multi-task (such as holding an unrelated telephoneconversation at the same time) and may allow others to use the system (such as askinga secretary to retrieve a document). Thus it was not possible to measure attentiondirectly.We may, however, infer that the Agent homepage (used significantly more by theExperimental group and providing interventions” (suggestions) only to the Experimentalgroup) is drawing the users’ attention to specific postings and other system elements.Indeed, it is the theoretical effects of attention management that led, ultimately, to theAtGentive enhancements, including the Agent home page.SatisfactionGenerally, participants stated that they were most satisfied with the fact that they hadaccess to documents and information about lecturers and other students. The Searchfacility was an important feature for locating these. The predominant source ofDel 4.4: AtGentive Final Evaluation Report page 72