Evaluating User Experience in Games: Concepts and Methods - Lirmm

x Foreword
personality which evolves and changes over time based on experience. This view
can be seen as hopeful. It says we become what we do.
In Chapter 10, Evaluating User Experience Factors Using Experiments:
Expressive Artificial Faces Embedded in Contexts, Lankes, Bernhaupt, and
Tscheligi present an experimental study investigating the effects of embodied conversational
agents in games on the overall user experience. This chapter shows that
it is necessary to conduct rigorous experiments to understand influencing factors on
the overall user experience.
In their chapter, Evaluating Exertion Games (Chapter 11), on the effect of exertion
in producing emotional engagement, Mueller and Bianchi-Berthouze explore
four methods for studying the relationship between exertion and user experience.
This is an important question from a research perspective, but the question gains
additional importance given the rise in popularity of activity-based games (e.g., Wii
sports and fit games) and the common criticism that most games are sedentary and
contribute to declining fitness in children. Like many of the other chapters, Mueller
et al. uses a case study approach testing four games and using different methods for
each one. Overall, the results are interesting and promising, albeit for some tests
opposite of what was expected. It would be early to standardize on any particular
approach. The chapter also points to the promise of automatic encoding of user
movements and physiological measures. Fortunately, Mueller et al. raises the often
underrated concern that any such measures fall short in capturing the contextual
factors which could be major determinants of emotional states and game experience.
In Chapter 12, Beyond the Gamepad: HCI and Game Controller Design and
Evaluation, Brown, Kehoe, Kirakowski, and Pitt focus on controllers as a determinant
of user experience in a game. They propose a McNamara and Kirakowski
framework that categorizes technology into experience, usability, and functionality.
They apply this model to the evaluation of three controllers: a steering wheel, the
game pad, and the keyboard. Three measures, completion time, subjective mental
effort questionnaire, and consumer product are used. They also do a content analysis
of user comments about each device classifying in terms of their valence (positive,
negative) and category (sensitivity, feedback, learning potential ...). Brown et al.
also captured behavioral data during the test, comparing the steering angle access
for the game pad and the steering for both experienced and inexperienced users. This
allows them to draw relatively sophisticated conclusions such as that for sophisticated
users the more sensitive the controller, the more useful the controller. This
is a good example of one of their main conclusions – that a complete collection of
measures combined with user comments enabled an analysis of causes and therefore
fixes.
In Chapter 13, Using Heuristics to Evaluate the Overall User Experience of Video
Games and Advanced Interaction, Koeffel, Hochleitner, Leitner, Haller, Geven, and
Tscheligi provide a comprehensive review of the recent literature on game evaluation
as an introduction to their approach of using heuristics to evaluate games. In
most cases, the heuristics refer to user experiences throughout the course of playing
the game. As such, they are user evaluation heuristics, e.g., the game outcome
should be perceived as fair. The heuristics do not specify ideal or minimal design
parameters, e.g., no more than five enemies should attack a player simultaneously.