Deliverable 4.4 - INSEAD CALT

More documents

Recommendations

Info

AtGentive IST-4-027529-STP - Attentive Agents for Collaborative Learners3. Tools for Summative EvaluationThis section describes the development of suitable tools – evaluation instruments – forthe summative evaluation of AtGentSchool and AtGentNet. Before development of thesetools begins, it is necessary to consider the user-groups and the tasks they will be askedto perform.3.1 Description of users and tasks3.1.1 Description of users - AtGentSchoolFive elementary schools in or near Prague in the Czech Republic participated in the usertrial, with either one or two classes per school. In some cases, only a portion of theregular class participated in the trial. Number of students participating per class thereforeranged from 6 to 14 per class, with 6 classes. The modal age of students was 11 years.Students had previous experience of working in groups, though not necessarily while atthe computer. The teachers assessed students as to their Ability, using the criteria ofCommunication Ability and Interest in the Outside World. Students were allocated intopairs for working on AtGentSchool. This was done by friendships, but in practice tendedto pair students of similar ability.The students worked in their native language (Czech). All teachers either spoke goodEnglish or had access to the appropriate translation / interpretation facilities. Facilitiesand infrastructure at the schools (such as computers and network connections) generallymet those anticipated of a contemporary UK school.3.1.2 Description of tasks - AtGentSchoolAtGentSchool is built on the Ontdeknet eLearning platform, created by the Dutchcompany Ontdeknet. This platform is an electronic learning environment that makesknowledge and skills in society accessible to educational institutions in general andindividual students in particular. Virtual learning relationships between subject expertsand students are established in this virtual learning environment. The Ontdeknetenvironment provides guidance to support individuals to learn together based uponcommon interests.The students were set the task of examining their home country (Czech Republic) and aforeign country (New Zealand) using AtGentSchool, with a view to deciding in whichcountry the students would prefer to live. A colleague teacher with experience of NewZealand acted as expert-traveller.3.1.3 Description of users - AtGentNetThe Swedish Trade Council (STC) runs a distance learning course for business peopleentitled TRIM (Trade Management Implementation). The TRIM project implements thenew International Trade Management (ITM) concept, devised by STC. This concept aimsto help SMEs (Small and Medium-sized Enterprises) to grow internationally. Participantsof the distance learning TRIM course are based in Namibia, South Africa, Slovenia,Norway, Iceland, Hungary and Sweden. The TRIM project ran until the end of October2007. The participants continue to have access to a “Virtual Trade Network” offering lifelong learning, career and business opportunities.Trainees within these seven countries comprised 46 managers who followed classes intheir own country and internationally as a group. The classes were on subjects aboutinternational business issues; examples include cross cultural training, changeDel <strong>4.4</strong>: AtGentive Final Evaluation Report page 20
AtGentive IST-4-027529-STP - Attentive Agents for Collaborative Learnersmanagement, market research, international seminars and technical expert knowledge.Trainees were supported by locally-based coaches / tutors who provided expert adviceand assistance.Each trainee collaborated with a tutor, working together for 0.5 days per month on whatwas termed the “coaching material”. This material consisted of questionnaires in differentsections, which the tutor and the manager selected and discussed together. Thesequestionnaires relate to the individual situation of each manager and his / her position ondifferent issues with respect to exporting; for example, the selection of the market andappropriate products. The tutors are trained in coaching skills and additional backgroundknowledge.3.1.4 Description of tasks - AtGentNetPrior to AtGentive, the IDCT platform had been used by the tutors and projectadministrators only to post information and to provide trainees with presentations of thelectures and supportive information, i.e. the platform was mostly used to storeinformation for the participants. While the AtGentive-modified platform does support thisform of use, through enhancement of perception, interventions added in relation toConceptual Framework-related scenarios require interactive use of the platform. It wastherefore necessary to create an extra task for the participants to work on that requiredinteraction (such as posting replies to documents).It is important to note that the managers involved in the TRIM project are very busy withtheir main work, and have difficulty finding extra time to participate. Therefore this taskconsisted of a business simulation exercise (see Section 2.2.7) which offered additionalexperience of business decision-making. Participants who chose to take part in this extraactivity were required to collaborate with each other, interaction necessary for the testingof AtGentNet interventions.3.2 Derivation of the Key IndicatorsThe evaluation process assesses the effectiveness of the AtGentive interventionsagainst a set of Key Indicators. The generation of the Indicators is described in detail indeliverable D4.3,3.3 Context, Interaction, Attitudes and OutcomesThe design for the evaluation plan incorporated an adaptation of an evaluationframework developed for the evaluation of Computer Aided Learning packages. TheCIAO! Framework (Jones et al., 1999) is based on the interaction between Context,Interaction, Attitudes and Outcomes. Context comprises the original aims and goals ofthe system designers. Interaction is the way in which students use the software –interplay between usability and activity. Attitudes and Outcomes are the results ofinteraction – effects on the students (such as frustration or being supported) anddesirable outcomes (such as improvements in learning or collaboration) (see Figure 10).• Context: the original aims and goals of the system designers. In the case ofAtGentive it is primarily the aims of interventions in relation to the scenario beingenacted.• Interaction: the way in which students use the software. This is an interactionbetween the software’s affordance (Gibson, 1979) (and thus usability) and theuser’s activity.Del <strong>4.4</strong>: AtGentive Final Evaluation Report page 21
Page 1 and 2: ATGENTIVEIST-4-027529-STPDELIVERABL
Page 3 and 4: AtGentive IST-4-027529-STP - Attent
Page 13: AtGentive IST-4-027529-STP - Attent
Page 22 and 23: May 2007AtGentive IST-4-027529-STP
Page 60 and 61: CollaborationAtGentive IST-4-027529
Page 76 and 77:
AtGentive IST-4-027529-STP - Attent
Page 78 and 79:
Page 80 and 81:
Page 82 and 83:
Page 84 and 85:
Page 86 and 87:
Page 88 and 89:
DescriptionAtGentive IST-4-027529-S
Page 90 and 91:
Integratability:AtGentive IST-4-027
Page 92 and 93:
Page 94 and 95:
Page 96 and 97:
Page 98 and 99:
Page 100 and 101:
Page 102 and 103:
Page 104 and 105:
Page 106 and 107:
Page 108 and 109:
Page 110 and 111:
Page 112 and 113:
Page 114 and 115:
Page 116 and 117:
Page 118 and 119:
Page 120 and 121:
Page 122 and 123:
Page 124 and 125:
Page 126 and 127:
Page 128 and 129:
Page 130 and 131:
Page 132 and 133:
Page 134 and 135:
Page 136 and 137:
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Page 154 and 155:
Page 156 and 157:
Page 158 and 159:
Page 160 and 161:
Page 162 and 163:
Page 164 and 165:
Page 166 and 167:
Page 168 and 169:
Page 170 and 171:
Page 172 and 173:
The effects of a computer agent’s
Page 174 and 175:
Page 176 and 177:
Page 178 and 179:
Page 180 and 181:
4. ResultsAtGentive IST-4-027529-ST
Page 182 and 183:
Page 184 and 185:
Additional pedagogical investigatio
Page 186 and 187:
Page 188 and 189:
Page 190 and 191:
Page 192 and 193:
Page 194 and 195:
Page 196 and 197:
Page 198 and 199:
Page 200 and 201:
Page 202 and 203:
Page 204 and 205:
Page 206 and 207:
Page 208:
show all

Deliverable 4.4 - INSEAD CALT

Create successful ePaper yourself

Delete template?

Save as template?