Emotion representation and physiology assignments in digital systems

More documents

Recommendations

Info

140 C. Peter, A. Herbon / Interacting with Computers 18 (2006) 139–170 1. Introduction The HCI community is becoming more and more aware of the importance of emotions in human–computer interaction. Especially in the usability domain there is a growing need and interest in considering the emotional ‘property’ of users (Reeves and Nass, 1996; Picard, 1997; Marcus, 2003; Cockton, 2004). Traditional analysis concepts like mental models (Carroll and Olson, 1988) or users’ models (Sasse, 1997) help to understand users’ approaches to problem solving and interaction with machines. Emotions, however, are barely covered by those theories. Also, product designers started to explore emotional aspects in their domain. Beginning with Pat Jordan’s and Don Norman’s considerations (Jordan, 2002; Norman, 2004), more and more contributions have been made to this field, like Pieter Desmet’s work (Desmet, 2002). In the field of human–computer interaction, Rosalind Picard’s work on affective computing has enormously increased the momentum of HCI-related emotion research (Picard, 1997; 1999). Today, there are several publications available on proof-of-possibility studies for emotion detection and affective response, for instance André et al. (2000); Kort et al. (2001); Klein et al. (2002); Picard and Klein (2002); Scheirer et al. (2002), and Arafa et al. (2004). Dedicated sensor systems and technology frameworks have been developed (Haag et al., 2004; Mader et al., 2004; Anttonen and Surakka, 2005; Peter et al., 2005), theoretical aspects have been addressed (Cañamero, 1999; Picard, 1999; Cockton, 2002; 2004; Hudlicka, 2003), and there are the first signs of a developing community interested in emotions and their role in HCI (see Marcus, 2003; HUMAINE, 2005; ACII, 2005; Peter and Blyth, 2005). However, all activities so far have been restricted to being just possibility studies and proof-of-concept prototypes without a clear theoretical foundation of emotions and how they should be dealt with by a digital system. Particularly, there exists no straightforward and applicable concept of structuring emotions, which is a prerequisite for reliably obtaining, storing, and finally processing emotion data. While there is a huge variety of emotion models in psychology, emotion related HCI research has so far widely neglected the need of an underlying theoretical model of emotions (cf. Cockton, 2004; Muller, 2004). Recently, some researchers have realised the need for an underlying theory for properly correlating physiological data 1 with emotional states. Several models of emotions developed by psychologists have been tried, like OCC (Ortony et al., 1988), or those from Scherer (1984); Frijda (1986), and Roseman et al. (1996). However, it is difficult to take a theory of one research field, like psychology or cognitive neuroscience, and apply it to another, like HCI. The same problem has been experienced by the HCI community in the late 1980s and 1990s, when cognitive processes were to be included in HCI research (cf. Rogers, 2004). Why should it be different with emotions? The emotion models developed by psychologists have been designed to study emotions in general. They do not just contain correlations between physiological and emotional states, but also different ideas on the cause of the arising emotion, underlying biological processes, anatomical structures, and other psychological considerations (cf. Davidson, 2003). Furthermore, 1 The term ‘physiological data’ here refers to any observable physiological changes, such as heart rate, skin resistance, skin temperature, mimics, gestures, posture, voice characteristics, and others.
C. Peter, A. Herbon / Interacting with Computers 18 (2006) 139–170 141 discussions regarding which model is eventually best suitable for measuring emotions are very controversial, not least because some authors even question the fundamentals of these models (cf. Fredrickson, 1998; Sabini and Silver, 2005). Implementing these models in software hence proved to be very difficult, and since the system developers were not psychologists, the resulting designs finally became simplified implementations of the original models and were adapted to the very specific tasks in mind. Obviously, there is not a straightforward way from psychology to HCI. Rather, designers and researchers in HCI should selectively use relevant and applicable knowledge gained by psychologists, taking the freedom to ignore aspects not relevant to their domain. HCI researchers are interested in other aspects of emotion than psychologists are, mainly in observable physiological manifestations of emotions occurring in real-life scenarios. Psychological emotion models do not live up to their requirements of applicability, comparability, and ease of use, as pointed out later in this paper. What is needed for HCI researchers and practitioners are adequate measures to associate physiological measurements to unambiguous emotional states in order to finally assign them to conditions meaningful to a computer (cf. Cockton, 2004; Bamidis et al., 2004; Wilson and Sasse, 2004; Ward and Marsden, 2004). The method described in this paper fulfils several requirements of HCI researchers and software developers on such a basis. It avoids artificial categorisation of emotions, requires no naming of emotional states, is language independent, and straightforward to implement. In the following section, an introductory overview of the two main models used to structure emotions is given and the supporting physiological findings are presented for each, from different studies. This is followed by a discussion of the disadvantages, shortcomings, and pitfalls of the discussed emotion models, from an HCI perspective. Based on this analysis, a new approach to structure emotion has been developed with a focus on its applicability in HCI research and application design. Results of an experiment based on this approach are described which show the applicability of the new approach. 2. Two main approaches to structure emotion Current theories from psychology on emotions can be grouped into theories that focus on how emotions arise and how they are perceived, and theories focussing on how observed emotions could be categorised or structured. Since theoretical aspects on how emotions arise, when and how they are perceived, and which biological mechanisms induce them are less important for systems to recognise emotions, these approaches would not be reviewed in this paper. Please refer to the respective literature (e.g. Scherer, 1984; Frijda, 1986; Roseman et al., 1996). Among the theories for categorising or structuring emotions, two main theories are currently established: a discrete approach, claiming the existence of universal ‘basic emotions’ (e.g. Plutchik, 1980; Ekman, 1992), and a dimensional approach, assuming the existence of two or more major dimensions which are able to describe different emotions and to distinguish between them (e.g. Russell, 1980). There is still controversy on the matter of which approach is the one that best captures the structure of emotion even though attempts have been made to conflate the two (Russell and Feldman Barrett, 1999).
Page 1: Interacting with Computers 18 (2006
Page 5 and 6: C. Peter, A. Herbon / Interacting w
Page 17 and 18: Appendix A. Psychophysiological stu
Page 19 and 20: Diast. BP Syst. BP Surprise HR Temp
Page 21 and 22: Pitch Contour Decrease Decrease (ph
Page 23 and 24: Interest Intensity F 0 floor/mean F

Emotion representation and physiology assignments in digital systems

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?