Deliverable 4.4 - INSEAD CALT

AtGentive IST-4-027529-STP - Attentive Agents for Collaborative Learnersdissatisfaction was that there was too much information visible at one time, with severalinterviewees assuming that much of the information was intended for “administrators”. Inaddition, they found it “hard to use” and “hard to navigate”. These factors were alsoidentified by the heuristic evaluation.However, no significant difference between the treatment groups was found. Thissuggests that while there are significant improvements that may be made to the platform,the problems encountered did not detract from using the AtGentive enhancements.LearningThe purpose of the ITM course is to provide an environment where business managerscan take steps to expend their company’s international trade. It is a practical, rather thanacademic, course (see Section 4.3.1.3). Learning that takes place is incidental (thoughcritical) to this process. Therefore, no specific learning is facilitated. This makes any formof “testing” of participants impractical.Furthermore, participants may not be aware of what they have learned. Nonetheless, theinterview questions were included for learning. The responses underline the practical andimplicit learning this course engenders. Participants spoke of learning about “differentcultures” and the importance of respecting and understanding others, as well as learning“Practical tools”.There is no evidence for any difference in learning between the treatment groups.4.3.2.3. ConclusionThe AtGentive enhancements to the ICDT platform appear to support either, or both,performance and collaboration, in the sense that support is provided for the activity –acquiring information or collaborating with other students. However, it must be the users’intention to perform these activities. The system was not found to promote such activity ifnot already the user’s intention.4.4 Overall conclusionThe AtGentive enhancements to the ICDT platform differed notably from those ofAtGentSchool. Both systems implemented three scenarios from the conceptualframework. For AtGentSchool – intended for primary school children – the activation ofthese scenarios by the agent was very effective. The students liked and responded to theagent, and performed better as a result.AtGentNet was designed for busy adults who typically fitted a few minutes use of theplatform into otherwise full schedules. It was anticipated that the agent-deliveredscenario approach may be problematic for these users (as indicated by one of theadditional experiments – see Section 5.1 – indeed, most participants did not visit theagent homepage or mention the agent in interviews or questionnaires.) Instead, theprimary changes to the ICDT platform for AtGentive was the support of perception – thatis, the subtle direction of attention by careful placement of on-screen elements.While the positive results seen from AtGentNet are thus likely to be due in main to themore subtle manipulation of attention, potential problems with this approach were found.Many of the participants reported over-complication of the interface and difficulties ofuse. This suggests that a more subtle approach is necessary and indicates the need formore research to define the boundary between beneficial attention direction throughperception and unhelpful information overload.Del 4.4: AtGentive Final Evaluation Report page 73