Deliverable 4.4 - INSEAD CALT

AtGentive IST-4-027529-STP - Attentive Agents for Collaborative LearnersPositive image of the animated characterThe user’s immediate reaction to seeing any animated character should be at leastneutral and preferably positive. The user should anticipate that the character’sappearance will make their task easier, not more difficult. The user should not havenegative feelings about the animated character (threatened, humiliated, etc.)User control and freedomThis is an extension to the “standard” heuristic. The user should feel in control of theAtGentive interventions. The user should not be worried that they will be interrupted atany moment, or that they are likely to miss something importantKey Indicator Four: LearningLearning experience is supportedInterventions should not cause the learning experience to be degraded. Where theintervention is intended to improve the learning experience, it should do so.Key Indicator Five: CollaborationCollaboration is supportedInterventions should not discourage collaboration. Where the intervention is intended toimprove collaboration, it should do so.3.6 QuestionnairesThe methodology followed for AtGentive questionnaire development was:Determine clearly what you want to measureGenerate an item poolDetermine the format of measurementHave the initial item pool reviewed by expertsConsider inclusion of validation itemsAdminister items to a development sampleEvaluate the itemsOptimise scale length(DeVellis, 2003. Chapter 5)We wished specifically to measure the Key Indicators (see Section 3.2) along withgeneral demographic information. The “item pool” (individual items to be measured)comprises those already identified in deliverable D4.3, a number of papers from previousstudies, general texts on surveys and further contemplation of the evaluationrequirements. This created a list of concepts to be measured. An example is shownbelow, with an abstract concept followed by a specific question. A full list is given inAppendix 2 and in Appendix 3.General (own) satisfaction with AtGentSchool [First day] [Bi-Weekly]I am completely satisfied with AtGentSchool (Agree completely, Agree, Agree alittle, Undecided, Disagree a little, Disagree, Disagree completely)3.6.1 AtGentSchoolQuestionnaires were administered to the students at the end of a class using thecomputers they used for AtGentive. The school environment restricted the number ofquestionnaires that could be fitted into class time. Two questionnaires were administeredDel 4.4: AtGentive Final Evaluation Report page 25