Deliverable 4.4 - INSEAD CALT

AtGentive IST-4-027529-STP - Attentive Agents for Collaborative Learners16 of the 17 pre-test questions, and 13 of the 15 post-test questions were found not tohave answers that differed significantly in correctness between the Control andExperimental groups.For the pre-test, question eight (“When it's winter in Czech Republic, it's winter in NZ aswell”) gave a significant result, with the Experimental group scoring better than theControl group.For the post-test, questions nine (“Most people live on South Island “) and 14 (“Thehighest building in the Southern hemisphere is in NZ”) gave significant results. TheExperimental group scored better than the Control group for question nine, whereas theControl group scored better than the Experimental group for question 14.4.2.2.4. Student Attitude Survey QuestionnairesStudents were asked to complete questionnaires twice during the pilot – mid-way andtowards the end (see Section 3.6.1). These questionnaires contained rating scales toassess the participants’ emotive response to AtGentSchool.The statements that students were asked to rate are as follows:1. Honza, the agent, helped me a lot2. The software does exactly what I want3. The software does not do anything that I want4. Honza looks great5. Honza is really friendly6. Honza is very helpful7. Honza is very annoying8. I think Honza likes me a lot9. When I use the keyboard or mouse, the software does something in responsestraight away10. I understand everything the software tells me I should do11. I like the look of the software12. I know what I’m doing when I use the software13. I feel in control of the software14. The instructions on the screen are really helpful15. I understood what the teacher told me to do16. I could quickly get hold of the teacher to ask questions (translated as “I have noproblem to ask teacher a question”)17. The teacher knew all about the software18. I really enjoyed the lesson19. How much time did you spend learning how to use AtGentSchool, compared toactually using it?20. We decided to divide the work equally21. We shared equally the typing on the computer22. We shared equally deciding what to type on the computerAlmost twice as many student pairs in the Control group completed the questionnairetwice compared to the Experimental group (22 compared to 14). The date at whichstudents completed these questionnaires varied considerably, both between and withinclasses. Since attitudes are affected by continued use of the system, the date eachstudent pair completes each questionnaire will be likely to affect the result. Comparisonsbetween student groups are more likely to be affected than comparison within studentgroups. Therefore a repeated measures approach was chosen.Del 4.4: AtGentive Final Evaluation Report page 47