Deliverable 4.4 - INSEAD CALT

More documents

Recommendations

Info

AtGentive IST-4-027529-STP - Attentive Agents for Collaborative LearnersNeutral Sad HappyConfused(Week1)Mann-Whitney U 237.000 252.500 349.500 320.000Wilcoxon W 643.000 658.500 755.500 698.000Z -2.499 -2.183 -.484 -1.069Asymp. Sig. (2-tailed) 0.012 0.029 0.629 0.285Table 6 – Analysis of feedbacks by treatment group (Mann-Whitney tests)Regarding the possible confounding variables (see the beginning of Section 4.2.2), noneof the feedback (“Neutral”, “Happy” and “Sad”) variables tested showed a significantdifference related to either gender 6 or ability.However, while “Confused” (for week one) was not significantly different by gender, itwas found to be significantly different for the Control group by Ability, as shown in Figure18 and Table 7.43.532.521.5CTLEXP10.50Low Medium SmartFigure 18 - "Confused" feedbacks during week one; mean values by treatment group andAbility6 Jonckheere-Terpstra test, looking for significance at the 0.05 level. This test was used instead ofKruskal-Wallis in order to take account of the Ability (LowMediumSmart) comprising anincreasing scaleDel <strong>4.4</strong>: AtGentive Final Evaluation Report page 42
AtGentive IST-4-027529-STP - Attentive Agents for Collaborative LearnersConfused, wk1 Control group Experimental groupNumber of Levels in Ability3 3N 27 28Observed J-T Statistic 58.500 85.500Mean J-T Statistic 108.000 122.500Std. Deviation of J-TStatistic19.184 21.633Std. J-T Statistic -2.580 -1.710Asymp. Sig. (2-tailed) .010 .087Table 7 - Jonckheere-Terpstra tests, showing significance of "Confused" for week one bytreatment group and AbilityInterventionsReconstruction of interventions for the Control groupIt was intended to reconstitute the interventions that the Control group would havereceived if they had been using the AtGentive-enhanced version of the software (as theExperimental group). That way, we could see how the interventions were related toevents. For example, if the experimental group went to the mind map on average 36seconds after a certain intervention, and the control group did the same thing 112seconds after an intervention would have been given to them, then we could say theintervention had an effect.In order to ensure that the reconstruction was correct, the following test was applied:1. Note that the Experimental group log files have lines of "Sending intervention"and the Control group log files do not2. Reconstruct the "Sending interventions" lines for both the Control group log filesand the Experimental group (even though the Experimental group already havethese lines)3. Check that the reconstructed "Sending intervention" lines for the Experimentalgroup are the same as the real lines for the Experimental group. If so, it providesan assurance that the reconstructed lines for the Control group are correctNone of the reconstitution methods tried passed this test. This was because an exactdefinition of the algorithms used by the reasoning module to initiate interventions for theexperimental group is required. Unfortunately, although the reasoning module deliveredappropriate interventions, in practice it did not exactly follow its original specification in away that is very difficult to reverse-engineer. This made it impractical to reconstruct“Sending intervention” lines for the control group that would be directly comparable withthe lines on the Experimental group’s log files. The intervention analysis therefore looksexclusively at interventions sent to the Experimental group.Interventions received and Marks obtainedAn analysis was made to look for any correlation between the number of interventionsstudents received and the marks they were awarded. Two significant correlations werefound (see Table 8), which are:Del <strong>4.4</strong>: AtGentive Final Evaluation Report page 43
Page 1 and 2: ATGENTIVEIST-4-027529-STPDELIVERABL
Page 3 and 4: AtGentive IST-4-027529-STP - Attent
Page 13: AtGentive IST-4-027529-STP - Attent
Page 22 and 23: May 2007AtGentive IST-4-027529-STP
Page 60 and 61: CollaborationAtGentive IST-4-027529
Page 88 and 89: DescriptionAtGentive IST-4-027529-S
Page 90 and 91: Integratability:AtGentive IST-4-027
Page 98 and 99:
AtGentive IST-4-027529-STP - Attent
Page 100 and 101:
Page 102 and 103:
Page 104 and 105:
Page 106 and 107:
Page 108 and 109:
Page 110 and 111:
Page 112 and 113:
Page 114 and 115:
Page 116 and 117:
Page 118 and 119:
Page 120 and 121:
Page 122 and 123:
Page 124 and 125:
Page 126 and 127:
Page 128 and 129:
Page 130 and 131:
Page 132 and 133:
Page 134 and 135:
Page 136 and 137:
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Page 154 and 155:
Page 156 and 157:
Page 158 and 159:
Page 160 and 161:
Page 162 and 163:
Page 164 and 165:
Page 166 and 167:
Page 168 and 169:
Page 170 and 171:
Page 172 and 173:
The effects of a computer agent’s
Page 174 and 175:
Page 176 and 177:
Page 178 and 179:
Page 180 and 181:
4. ResultsAtGentive IST-4-027529-ST
Page 182 and 183:
Page 184 and 185:
Additional pedagogical investigatio
Page 186 and 187:
Page 188 and 189:
Page 190 and 191:
Page 192 and 193:
Page 194 and 195:
Page 196 and 197:
Page 198 and 199:
Page 200 and 201:
Page 202 and 203:
Page 204 and 205:
Page 206 and 207:
Page 208:
show all

Deliverable 4.4 - INSEAD CALT

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?