Evaluating User Experience in Games: Concepts and Methods - Lirmm

More documents

Recommendations

Info

250 C. Koeffel et al. calculating a weighted average. Their score reaches from 0 to 100 and can therefore be seen as a percentage rating which is very common among reviewing sites. Unfortunately, Metacritic.com did not have a rating for “Die Siedler: Aufbruch der Kulturen”. Therefore, we gathered 13 different review scores from several review pages through our own research and calculated the average rating. The resulting ranking of our study can be seen in Table 13.2. It shows that the sequence of the games evaluated according to process described above is similar to the sequence obtained from Metacritic.com. This tendency shows the connection between heuristic evaluations and user experience (which is the main focus of the review from Metacritic). Especially, the ranking according to usability issues found during the evaluation appears to comply with the user experience-based results from Metacritic.com. In relation to the results from Metacritic.com, we can state that the more the usability issues are found during a heuristic evaluation, the worse the user experience is. The fact that the ranking according to points is not as high as the ranking according to Metacritic.com can be caused by the fact that our heuristics focus on usability issues which might not be detected during a game review or which might not be weighted that dramatically. On the other hand, we also acknowledge that the testing time of about 2 hours per game was most likely not long enough to achieve total immersion. Therefore, certain effects as described in Cheng and Cairns (2005) such as overlooking usability issues when being totally immersed did not occur. Nevertheless, to further prove this concept more extensive evaluations (with more games from different genres and for longer periods of time) are proposed. This could also lead to a definite number of heuristics that have to be fulfilled in order to grant an optimised user experience. In order to combine research conducted both in the area of video games and tabletop applications and to complete the framework as introduced in this section, Koeffel and Haller introduce 10 heuristics for the development of tabletop games, which are described in more detail in the next section (Koeffel and Haller 2008). 13.7 A Framework of Heuristics for the Evaluation of a Tabletop Game’s User Experience While the heuristics introduced in Section 13.6 are very well suited for most video games such as standard platform PC or console games, they lack tabletop-specific aspects. According to Jegers (2008), pervasive games differ in many ways from traditional computer games. This principle also applies to tabletop games, especially since they include new social and physical experiences as described in Section 13.1. Therefore, within this section a set of device-specific heuristics will be introduced, which include the social issues as pointed out in Jegers (2008), Sweetser and Wyeth (2005) as well as particular issues that arise when interacting with a tabletop game. This section completes the aforementioned framework (see Section 13.6) with tabletop-specific aspects, some of which might seem similar to other heuristics in the framework at first. Nevertheless, the here selected 10 heuristics are tailored to tabletop games and can profoundly influence experience as well as the usability
13 Using Heuristics to Evaluate the Overall User Experience 251 of such a game. To emphasise this aspect, these heuristics are deliberately placed in the device-specific part of the framework. To achieve a set of rules as complete as possible, an iterative approach based on existing tabletop-related work has been chosen. Our approach, first described in Koeffel (2007), began with the creation of a first set of 12 heuristics, based on existing literature work and research trials in the field of usability. After additional research, these heuristics have been checked for weaknesses, refined and paraphrased. The second iteration focused on comprehensibility and tried to better fulfil the original concept of heuristics as “rules of thumb”. Additional feedback from experts initiated the creation of a third iteration of heuristics. These now 11 rules were used to conduct a heuristic evaluation. Six different games were evaluated. Twelve participants (3 usability experts, 3 double experts, 3 game experts and 3 medium experienced users) were presented with the 11 heuristics from the third set and then instructed to play each of the games. They were asked to identify problems and assign them to the matching heuristics afterwards. The results and findings of this evaluation led to the final version of our proposed heuristics for tabletop games. This final set of tabletop-specific heuristics emanated from the third set after the heuristic evaluation was conducted. It saw an inclusion of subcategories to further aid evaluators by clarifying potential obscurities. Table 13.3 shows the final heuristics we achieved: A full description of all 10 heuristics (illustrated with pictures to display occurring issues) can be found in Koeffel (2007). No. Heuristic Table 13.3 Device-specific heuristics for tabletop games 1 Cognitive workload: The cognitive workload which is not connected to the game play (i.e., in connection with the acquisition of skills, the view, the screen orientation and the input methods) should be minimised. 2 Challenge: The system should be designed in a way that the challenge satisfies the preconditions of a tabletop setup and the target group. 3 Reach: The reach of the players should be adapted to the requirements of the game play. 4 Examinability: The players should not be hindered to examine the area required by the game play. 5 Adaptability: The system should be adaptable to the player in terms of the setup. 6 Interaction: The interaction method should satisfy the expectations of the player and follow the game logic. 7 Level of automation: The player should be able to execute all actions relevant to the game by herself. 8 Collaboration and communication: The interpersonal communication and collaboration should be supported by the entirety of the game (such as game play and setup). 9 Feedback: Feedback and feedthrough should be adapted to the possibilities of tabletop games, used adequately and provided to the players when appropriate. 10 Comfort of the physical setup: The construction of the setup (including the display) should be comfortable to use and not require the player to take an awkward position.
Page 2 and 3:
Human-Computer Interaction Series E
Page 4 and 5:
Regina Bernhaupt Editor Evaluating
Page 6:
For Philippe
Page 9 and 10:
viii Foreword Chapter 3, Presence I
Page 11 and 12:
x Foreword personality which evolve
Page 14 and 15:
Contents Part I Introduction to Eva
Page 16 and 17:
Contributors Regina Bernhaupt IRIT,
Page 18:
Contributors xvii Manfred Tscheligi
Page 21 and 22:
xx About the Authors Emily Brown is
Page 23 and 24:
xxii About the Authors a post-docto
Page 25 and 26:
xxiv About the Authors Christina Ko
Page 27 and 28:
xxvi About the Authors MIT Media La
Page 29:
xxviii About the Authors Theft Auto
Page 33 and 34:
Chapter 1 User Experience Evaluatio
Page 35 and 36:
1 User Experience Evaluation in Ent
Page 37:
1 User Experience Evaluation in Ent
Page 41 and 42:
Chapter 2 Enabling Social Play: A F
Page 43 and 44:
2 Enabling Social Play 13 It is als
Page 45 and 46:
2 Enabling Social Play 15 observati
Page 47 and 48:
2 Enabling Social Play 17 for playe
Page 49 and 50:
2 Enabling Social Play 19 to provid
Page 51 and 52:
2 Enabling Social Play 21 Colwell J
Page 53 and 54:
Chapter 3 Presence, Involvement, an
Page 55 and 56:
3 Presence, Involvement, and Flow i
Page 57 and 58:
Page 59 and 60:
Page 61 and 62:
Page 63 and 64:
Page 65 and 66:
Page 67 and 68:
Page 69 and 70:
Page 71 and 72:
Page 73 and 74:
Page 75 and 76:
Page 77 and 78:
Chapter 4 Assessing the Core Elemen
Page 79 and 80:
4 Assessing the Core Elements of th
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101:
Page 104 and 105:
74 E. Brown (2005). Björk and Holo
Page 106 and 107:
76 E. Brown 5.2.3 How We Work Toget
Page 108 and 109:
78 E. Brown could be simulated on p
Page 110 and 111:
80 E. Brown 5.3.2 Pre-production At
Page 112 and 113:
82 E. Brown from the code. This is
Page 114 and 115:
84 E. Brown At this point, there is
Page 116 and 117:
86 E. Brown Game developers have be
Page 119 and 120:
Chapter 6 Investigating Experiences
Page 121 and 122:
6 Investigating Experiences and Att
Page 123 and 124:
Page 125 and 126:
Page 127 and 128:
Page 129 and 130:
Page 131 and 132:
Page 133 and 134:
Page 135:
Page 138 and 139:
108 G. McAllister and G.R. White Th
Page 140 and 141:
110 G. McAllister and G.R. White su
Page 142 and 143:
112 G. McAllister and G.R. White of
Page 144 and 145:
114 G. McAllister and G.R. White te
Page 146 and 147:
116 G. McAllister and G.R. White qu
Page 148 and 149:
Page 150 and 151:
120 G. McAllister and G.R. White 7.
Page 152 and 153:
Page 154 and 155:
124 G. McAllister and G.R. White an
Page 156 and 157:
126 G. McAllister and G.R. White Al
Page 158 and 159:
128 G. McAllister and G.R. White Ne
Page 161 and 162:
Chapter 8 User Experience Design fo
Page 163 and 164:
8 User Experience Design for Inexpe
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177:
Page 180 and 181:
150 K. Poels et al. in games. This
Page 182 and 183:
152 K. Poels et al. their game char
Page 184 and 185:
154 K. Poels et al. If we apply thi
Page 186 and 187:
156 K. Poels et al. Participants ge
Page 188 and 189:
158 K. Poels et al. 9.3.2 Exploring
Page 190 and 191:
160 K. Poels et al. If our train is
Page 192 and 193:
162 K. Poels et al. would be intere
Page 195 and 196:
Chapter 10 Evaluating User Experien
Page 197 and 198:
10 Evaluating User Experience Facto
Page 199 and 200:
Page 201 and 202:
Page 203 and 204:
Page 205 and 206:
Page 207 and 208:
Page 209 and 210:
Page 211 and 212:
Page 213:
Page 217 and 218:
Chapter 11 Evaluating Exertion Game
Page 219 and 220:
11 Evaluating Exertion Games 189 pr
Page 221 and 222:
11 Evaluating Exertion Games 191 th
Page 223 and 224:
11 Evaluating Exertion Games 193 to
Page 225 and 226:
11 Evaluating Exertion Games 195 co
Page 227 and 228:
11 Evaluating Exertion Games 197 11
Page 229 and 230: 11 Evaluating Exertion Games 199 an
Page 231 and 232: 11 Evaluating Exertion Games 201 11
Page 233 and 234: 11 Evaluating Exertion Games 203 to
Page 235 and 236: 11 Evaluating Exertion Games 205 Ac
Page 237: 11 Evaluating Exertion Games 207 Pa
Page 240 and 241: 210 M. Brown et al. challenges that
Page 242 and 243: 212 M. Brown et al. was also happen
Page 244 and 245: 214 M. Brown et al. Another importa
Page 246 and 247: 216 M. Brown et al. Albinsson and Z
Page 248 and 249: 218 M. Brown et al. asked to report
Page 250 and 251: 220 M. Brown et al. Table 12.2 Mean
Page 252 and 253: 222 M. Brown et al. Table 12.7 Cont
Page 254 and 255: 224 M. Brown et al. Table 12.9 Cont
Page 256 and 257: 226 M. Brown et al. Count 450 400 3
Page 258 and 259: 228 M. Brown et al. feedback. Thus,
Page 260 and 261: 230 M. Brown et al. Desurvivre H, C
Page 263 and 264: Chapter 13 Using Heuristics to Eval
Page 265 and 266: 13 Using Heuristics to Evaluate the
Page 279: 13 Using Heuristics to Evaluate the
Page 287 and 288: Index A Advanced interaction games,
show all

Evaluating User Experience in Games: Concepts and Methods - Lirmm

Create successful ePaper yourself

Delete template?

Save as template?