Evaluating User Experience in Games: Concepts and Methods - Lirmm

More documents

Recommendations

Info

196 F. Mueller and N. Bianchi-Berthouze as players answer questions regarding their experience after they have played the game, “outside” their immediate engagement with it, a critique in common with post-interviews. The participants need to divert their focus of attention to the evaluation task, diverting from the experience; the same experience they are now asked to self-assess. This criticism is common amongst questionnaire approaches, whether the game facilitates exertion or not. However, if the players exerted themselves as part of the game play, several factors influence their answers in ways different to a keyboard or gamepad experience: First, as exertion games are believed to facilitate more emotional play (Bianchi-Berthouze et al. 2007), these affected states of emotions could influence the assessment players give, in particular if the questions are asked immediately after the game. We acknowledge that these altered emotional states could be a desired effect of the game, worth capturing in the evaluation process; however, we want to point out that researchers should be aware that the emotional change could occur not only from the game content, but also from the physical exertion the game facilitates, which might have different implications for the analysis. Second, a possibly lower recovery curve from a heightened state of arousal based on exertion might affect an effective comparison with non-exertion game data. To explain: The emotions facilitated by the involvement of the body interact with the physiological functions of the body in a bidirectional relationship, and it has been suggested that this relationship can affect the emotional engagement with the game for longer than in a traditional non-exertion game, in which the engagement is mainly regulated by cognitive functions (Lehrer 2006). This prolonged engagement with the game is not limited to emotional aspects, for example in our investigations of Breakout for Two (Mueller et al. 2003), we have observed that players needed a break to physically recover from the activity before they were able to fill out a questionnaire. This suggests that the exertion aspect can affect the time between the game experience and answering questions, possibly altering the recall capability of the experience. Furthermore, research has shown that cognitive functions are improved after exercise (Ratey 2008), which might also impact upon how the subjects answer, independent from the experience under investigation. These potential effects do not eradicate the use of questionnaires as evaluation method for exertion games, but researchers might benefit from being aware of these potential influences that are quite different than what is expected in traditional questionnaire experiences in order to address them in their evaluation design. 11.4 Evaluating User Experience In-Place In the previous sections, we have focused on methods that rely on data gathered after the gaming action. We are now describing our experiences with directly observing exertion actions while they are taking place. We focus on how the analysis of participants’ non-verbal behaviour can give insights into their experience, in particular, we describe how it helped us to quantify and reason about the effect of a game’s design on social and emotional experiences.
11 Evaluating Exertion Games 197 11.4.1 Coding Body Movement Case study: Donkey Konga. We carried out a study to investigate how the use of whole body game controllers would change the way players engage in a game (Lindley et al. 2008). An experiment was thus designed to observe and compare the behaviour of players playing the same game but using different types of controllers: controllers that require only finger movement to control the game and controllers that require larger body movement. Experimental setup. Levels of engagement and the degree of emotional and social interactions between players were explored in a game of Donkey Konga (Lindley et al. 2008). The input devices were bongos and a standard dual-pad controller. When bongos were used, players were encouraged to tap the bongos and clap their hands in time with the music; when the dual-pad controller was used, these actions were performed through button pressed using fingers and thumbs. We are aware that playing augmented bongos does not necessarily result in intense physical exhaustion; however, the involved body movements and their reliance on rhythmic coordinated kinesthetic actions have many characteristics similar to sportive behaviour and have been previously compared to exertion games (Bogost 2005), and hence the results should be able to contribute to an understanding of movement-based activities in general. Ten pairs of participants were asked to play in both conditions, and the order of the two conditions was counterbalanced across the pairs. Being all beginners, the players played in two-player cooperative mode (“Duet”) at the easiest skill setting. The playing sessions were videotaped and an existing engagement questionnaire (Chen et al. 2005) was used. The scores for the participants in each pair were summed. To measure the emotional and social engagements of the participants, their verbal and non-verbal behaviours were coded using the Autism Diagnostic Observation Schedule (Lord et al. 2000). We found this scheme particularly useful; however, other researchers seem to prefer Laban’s notations, especially when concerned with dance-like movements (Loke et al. 2007). The length of time that each participant spent producing speech and other utterances was measured. Non-verbal behaviours were also classified according to two categories: Instrumental gestures were defined as those in which the action conveyed a clear meaning or directed attention (e.g., pointing, shrugging, and nods of the head); empathic gestures were defined as those in which the action was emotive (e.g., placing the hands to the mouth in shock). These gestures were selected as they indicate the players’ social and emotional involvements. Analysis and discussion: To understand the magnitude of the effect the body movements has on the players, a statistical analysis of the non-verbal and verbal behaviours was performed. Prior to this, scores on the game were compared across the two conditions to ensure that possible effects were not due to variations in performance. Wilcoxon’s two-tailed matched-pairs signed-ranks test showed that the type of controller had no significant effect on performance (Z = –0.889, p = 0.414). All further differences were evaluated for statistical significance using Wilcoxon’s one-tailed matched-pairs signed-ranks tests, with the pair as the sampling unit.
Page 2 and 3:
Human-Computer Interaction Series E
Page 4 and 5:
Regina Bernhaupt Editor Evaluating
Page 6:
For Philippe
Page 9 and 10:
viii Foreword Chapter 3, Presence I
Page 11 and 12:
x Foreword personality which evolve
Page 14 and 15:
Contents Part I Introduction to Eva
Page 16 and 17:
Contributors Regina Bernhaupt IRIT,
Page 18:
Contributors xvii Manfred Tscheligi
Page 21 and 22:
xx About the Authors Emily Brown is
Page 23 and 24:
xxii About the Authors a post-docto
Page 25 and 26:
xxiv About the Authors Christina Ko
Page 27 and 28:
xxvi About the Authors MIT Media La
Page 29:
xxviii About the Authors Theft Auto
Page 33 and 34:
Chapter 1 User Experience Evaluatio
Page 35 and 36:
1 User Experience Evaluation in Ent
Page 37:
1 User Experience Evaluation in Ent
Page 41 and 42:
Chapter 2 Enabling Social Play: A F
Page 43 and 44:
2 Enabling Social Play 13 It is als
Page 45 and 46:
2 Enabling Social Play 15 observati
Page 47 and 48:
2 Enabling Social Play 17 for playe
Page 49 and 50:
2 Enabling Social Play 19 to provid
Page 51 and 52:
2 Enabling Social Play 21 Colwell J
Page 53 and 54:
Chapter 3 Presence, Involvement, an
Page 55 and 56:
3 Presence, Involvement, and Flow i
Page 57 and 58:
Page 59 and 60:
Page 61 and 62:
Page 63 and 64:
Page 65 and 66:
Page 67 and 68:
Page 69 and 70:
Page 71 and 72:
Page 73 and 74:
Page 75 and 76:
Page 77 and 78:
Chapter 4 Assessing the Core Elemen
Page 79 and 80:
4 Assessing the Core Elements of th
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101:
Page 104 and 105:
74 E. Brown (2005). Björk and Holo
Page 106 and 107:
76 E. Brown 5.2.3 How We Work Toget
Page 108 and 109:
78 E. Brown could be simulated on p
Page 110 and 111:
80 E. Brown 5.3.2 Pre-production At
Page 112 and 113:
82 E. Brown from the code. This is
Page 114 and 115:
84 E. Brown At this point, there is
Page 116 and 117:
86 E. Brown Game developers have be
Page 119 and 120:
Chapter 6 Investigating Experiences
Page 121 and 122:
6 Investigating Experiences and Att
Page 123 and 124:
Page 125 and 126:
Page 127 and 128:
Page 129 and 130:
Page 131 and 132:
Page 133 and 134:
Page 135:
Page 138 and 139:
108 G. McAllister and G.R. White Th
Page 140 and 141:
110 G. McAllister and G.R. White su
Page 142 and 143:
112 G. McAllister and G.R. White of
Page 144 and 145:
114 G. McAllister and G.R. White te
Page 146 and 147:
116 G. McAllister and G.R. White qu
Page 148 and 149:
Page 150 and 151:
120 G. McAllister and G.R. White 7.
Page 152 and 153:
Page 154 and 155:
124 G. McAllister and G.R. White an
Page 156 and 157:
126 G. McAllister and G.R. White Al
Page 158 and 159:
128 G. McAllister and G.R. White Ne
Page 161 and 162:
Chapter 8 User Experience Design fo
Page 163 and 164:
8 User Experience Design for Inexpe
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176: 8 User Experience Design for Inexpe
Page 177: 8 User Experience Design for Inexpe
Page 180 and 181: 150 K. Poels et al. in games. This
Page 182 and 183: 152 K. Poels et al. their game char
Page 184 and 185: 154 K. Poels et al. If we apply thi
Page 186 and 187: 156 K. Poels et al. Participants ge
Page 188 and 189: 158 K. Poels et al. 9.3.2 Exploring
Page 190 and 191: 160 K. Poels et al. If our train is
Page 192 and 193: 162 K. Poels et al. would be intere
Page 195 and 196: Chapter 10 Evaluating User Experien
Page 197 and 198: 10 Evaluating User Experience Facto
Page 213: 10 Evaluating User Experience Facto
Page 217 and 218: Chapter 11 Evaluating Exertion Game
Page 219 and 220: 11 Evaluating Exertion Games 189 pr
Page 221 and 222: 11 Evaluating Exertion Games 191 th
Page 223 and 224: 11 Evaluating Exertion Games 193 to
Page 225: 11 Evaluating Exertion Games 195 co
Page 229 and 230: 11 Evaluating Exertion Games 199 an
Page 231 and 232: 11 Evaluating Exertion Games 201 11
Page 233 and 234: 11 Evaluating Exertion Games 203 to
Page 235 and 236: 11 Evaluating Exertion Games 205 Ac
Page 237: 11 Evaluating Exertion Games 207 Pa
Page 240 and 241: 210 M. Brown et al. challenges that
Page 242 and 243: 212 M. Brown et al. was also happen
Page 244 and 245: 214 M. Brown et al. Another importa
Page 246 and 247: 216 M. Brown et al. Albinsson and Z
Page 248 and 249: 218 M. Brown et al. asked to report
Page 250 and 251: 220 M. Brown et al. Table 12.2 Mean
Page 252 and 253: 222 M. Brown et al. Table 12.7 Cont
Page 254 and 255: 224 M. Brown et al. Table 12.9 Cont
Page 256 and 257: 226 M. Brown et al. Count 450 400 3
Page 258 and 259: 228 M. Brown et al. feedback. Thus,
Page 260 and 261: 230 M. Brown et al. Desurvivre H, C
Page 263 and 264: Chapter 13 Using Heuristics to Eval
Page 265 and 266: 13 Using Heuristics to Evaluate the
Page 277 and 278:
13 Using Heuristics to Evaluate the
Page 279 and 280:
Page 281 and 282:
Page 283 and 284:
Page 285 and 286:
Page 287 and 288:
Index A Advanced interaction games,
show all

Evaluating User Experience in Games: Concepts and Methods - Lirmm

Create successful ePaper yourself

Delete template?

Save as template?