EMA: A computational model of appraisal dynamics - University of ...

Appears in Agent Construction and Emotions, 2006EMA: A computational model of appraisal dynamicsStacy MarsellaUniversity of Southern Californiamarsella@isi.usc.eduJonathan GratchUniversity of Southern Californiagratch@ict.usc.eduAbstractA computational model of emotion must explain both the rapid dynamics of some emotional reactions aswell as the slower responses that follow deliberation. This is often addressed by positing multiple appraisalprocesses such as fast pattern directed vs. slower deliberative appraisals. In our view, this confusesappraisal with inference. Rather, we argue for a single and automatic appraisal process that operatesover a person’s interpretation of their relationship to the environment. Dynamics arise from perceptualand inferential processes operating on this interpretation (including deliberative and reactive processes).We illustrate this perspective through the computational modeling of a naturalistic emotional situation.1 IntroductionChange is an inherent quality of emotion. Aroused byan unpleasant event, a person might explode into anger,fume at a slow boil or collapse into a depression.Once aroused, emotions influence our actions andjudgments concerning the event, altering what Lazarus[1] calls the person-environment relationship. Changesto this relationship can induce new emotional responses,resulting in a cycle of change in the person’srelation to the environment. These changes can berapid, on the order of milliseconds, or unfold overdays and weeks. In short, emotions are inherently dynamic,linked to both the world’s dynamics and thedynamics of the individual’s physiological, cognitiveand behavioral processes.A key challenge for any theory of emotion is to capturethis dynamic emotional process. Over the last 50years appraisal theories have been established as aleading theory of emotion. These theories argue thatemotion arises from a person’s interpretation of theiroverall relationship with the environment as characterizedby a set of appraisal variables (e.g., is this eventdesirable, who caused it, what power do I have overits unfolding). To date, however, appraisal theorieshave largely focused on structural considerations(e.g., specifying the components of appraisal) [2]. Farless progress has been made in detailing the processesthat underlie appraisal, with some notable exceptions.For example, Lazarus proposed a cyclical relationshipbetween appraisal, coping and re-appraisal. At a finergrain, Scherer’s sequential checking model argues forthe sequencing of distinct appraisal checks (relevance,etc.) that determine the appraisal variables [3]. Smithand Kirby argues for a two process model of appraisalwhereby associative processing, a memory-basedprocess and reasoning, a slower and more deliberativeprocess, operate in parallel [4]. Moors investigatesautomaticity of some appraisal processes [5]. Reisenzeinproposed a model of appraisal that distinguishesbetween hardwired appraisal processes for some appraisaldimensions and peripheral appraisals that canbe deliberative or learned [2].However, further progress is needed if appraisaltheories are to provide a full account of emotionalprocesses and their role in behavior. Specifically, theoriesmust detail the processes by which each appraisalvariable is determined, including the logical and temporaldependencies between these appraisal processes[4]. Additionally, the basic calculus of how the resultsof appraisal result in emotions of varying types, intensitiesand durations must be determined. Completingthe cycle, the impact of emotions on coping responsesand subsequent changes in the environment-personrelationship must be detailed.We see computational models of emotion as a powerfulapproach to concretizing and exploring the dynamicproperties of appraisal. The construction of acomputational model forces specific commitmentsabout how the person-environment relationship is represented,how appraisals are performed on those representations,the role of perception, memory, interpretationand inference in appraisal, and the relationshipbetween appraisals, emotions and coping responses.Often these commitments raise issues that are unforeseenat more abstract specifications of a theory. Further,once computationally realized, simulation allowsthe model to be systematically explored and manipulated,thereby generating predictions that can be furthertested with human subjects. Indeed given thecomplexity of appraisal theories, exploring dynamicproperties of a theory and contrasting alternative theoriesfrom a process-based perspective may arguablyhinge on their computer realization.This paper advocates a theoretical stance towardsthe problem of capturing emotional dynamic informedboth by the appraisal theory of Smith and Lazarus [6]and our experience in realizing this theory in a computationalmodel called EMA (EMotion and Adaptation).In our view, process theories of appraisal have con-

Frame 2 Frame 5 Frame 9 Frame 22Frame 60 Frame 80 Frame 272Figure 1flated appraisal and inference. Rather, we argue for asingle and automatic appraisal process that operates inparallel over a person’s interpretation of their relationshipto the environment. Dynamics arises from perceptualand inferential processes operating independentlyover this interpretation (including deliberative andreactive processes). We illustrate this perspective bymodeling of a naturalistic emotional situation in EMA.An example of emotion dynamics.One approach to studying emotion dynamics is to useassessment tools that ask a subject to imagine an evolvingsituation and to introspect on their emotional reactions[7]. This approach has in turn been leveraged toevaluate computer-based models of emotion [8].However, the focus in this paper is different. Insteadof a slowly evolving situation, our interest here is inan evocative situation that elicits a wide array of emotionalresponses over a short time period. Specifically,we will analyze and model a naturalistic emotioninvokingsituation recorded fortuitously during one ofour lab studies. We were videotaping actors at 30frames per second as part of a study on gestures andpostures. In the midst of instructing the actors, a pigeonunexpectedly flew in window. Figure 1 revealsthe reactions of one of the two actors. Although suchunexpected, uncontrolled event makes a rigorousanalysis problematic, this naturalistic setting serveswell to illustrate the rapid dynamics that we wouldwant to cover in a process model of appraisal.In the video, the actor holding the umbrella goesthrough a sequence of behaviors that suggest the followinginterpretation:• surprise at an unexpected event (frame 2),• fear (12),• an aggressive stance of self-protection (13-23),• relaxation (29),• concern for others (29-42), specifically for the birdthat caused the initial negative reaction and, finally,• an active helping strategy (62-80) combined with relaxedfacial features and smiling suggestive of relief.The sequence of behaviors that suggest that interpretationis as follows. By frame 2 (F2), the actor has begun to turnand orient toward the sound of the bird. Her eyebrowsrise (F3 through F4). The eyebrows return to a more neutrallevel and the mouth begins to open by F8. The Eyebrowslower and the jaw then drops during F11 and F12.In F13, she begins to grab the umbrella at the base, movethe left foot back away from bird and starts to raise arms.She raises the umbrella (F14 through F22), shifting her

weight to her right, rear foot away from the bird. Herposture and grasp of the umbrella suggests she is preparedto ward off a presumed attack of the bird bywhacking it with the umbrella. She continues her backwardmotion. Her motions slow and by F29 her left handstarts to let go of the umbrella and move towards hermouth. The umbrella is lowered in F34 and her left handcovers her mouth by frame F42. By F62 the backwardmotion stops (she moves approximately 6 feet) and theleft hand begins to lower from covering her mouth. ByF66, the actor begins to move forward and the hand lowerssufficiently to reveal relaxed facial features. In F72through F80, the forward motion continues, the handforms into a stop gesture and the face appears to be smiling(laughter and utterances expressing concern for thebird are also heard).A seemingly identical sequence of reactions is visiblein the other actor: raised eyebrows, lowered eyebrowsand jaw drop, followed by expressions suggestingrelief/amusement and compassion. But reactionsalso differ, for she becomes aware of the bird later, sheis closer to the threat and certain responses are notfacilitated by the instrumentality of the umbrella.This rapid transition in the actor’s expressive stateand behaviors lasts 2.6 seconds. The expression ofraised eyebrows often associated with surprise takeson the order 30-60 milliseconds and the expression oflowered eyebrows and lowered jaw often associatedwith anger and responses to threat takes on the order300 milliseconds. Overall, this suggests an interestingprogression from an appraisal of unexpectedness, to anassessment of personal significance, and finally anappraisal of significance to others (cf. Scherer’s sequentialchecking, [3]). Tightly coupled with theseappraisal dynamics from threat-to-self to threat-tootherand emotion dynamics of Fear/Anger to Compassion/Relief,there is a corresponding progression ofcoping responses from defend/attack to help.Several factors can help us explain these dynamics.The process of appraising the situation in terms of itsunexpectedness, congruency with the actor’s concerns,etc. can have its own internal dynamics grounded insuch factors as inferential demands that underlie theappraisal and/or potential logical ordering relationsbetween steps of the appraisal. As Scherer’s theory ofappraisal checks argues, there may be a temporalcourse to the appraisal process. There are also perceptualprocesses and inferential processes that alter theactor’s interpretation of the situation appraisal. Similarlythere are the processes of forming a coping responseor plan to deal with the event. Finally, thesituation and subsequent re-appraisals occur due tochanges as the event unfolds. This unfolding occursboth due to the actor’s response of “arming herself”and moving away from the event as well as other entitiesbecoming the focus (such as the threat to the bird).Basic Theoretical AssumptionsA central tenant in cognitive appraisal theories in general,and Smith and Lazarus’ work in particular, is that appraisaland coping center around a person’s interpretationof their relationship with the environment. This interpretationis constructed by cognitive processes, summarizedby appraisal variables and altered by coping responses.The goal of our work is to develop a processmodel of appraisal, realized as a computational process.In our view, there are several key challenges for acomputational model of appraisal. The model mustexplain both the rapid dynamics of some emotionalreactions as well as the slower evolution of emotionalresponses that may follow some deliberation and inferences.The appraisal processes must in some way beconsistent with both these reactive and deliberativeresponses. In addition, appraisal processes must operateover a range of phenomena spanning simple physicalphenomena as well as complex social situations.These challenges are often addressed by presumingmultiple appraisal processes, for example fast patterndirected appraisal processes and slower more deliberativeappraisals (e.g., [2, 4, 5]). Our theoretical stanceon this issue differs considerably.First and foremost, we cleanly delineate betweenappraisal and inference. We argue that appraisal processesare always fast (reactive), parallel and unique inthe sense that we postulate a single process for eachappraisal variable. However, multiple other perceptualand cognitive processes perform inference (both fastand slow, both deliberative and reactive) over the interpretationof the person-environment relationship. Asthose inference processes change the interpretation,they indirectly trigger automatic reappraisal.Thus, debates about which emotions have a cognitiveor non-cognitive basis become moot. The relationbetween cognition and appraisal is that appraisals operateon the results of cognitive operations as well asany other operation that transforms the person’s interpretationof their relationship to the environment.A computational model of appraisalEMA is a computational model that realizes this theoreticalstance. We now sketch the basic outlines of the model(See [9] for a more complete description.) In generalterms, we characterize a computational model as a processor processes operating on representations. A computationalmodel of appraisal is a set of processes that interpretand manipulate a representation of the personenvironmentrelationship.In our view, a core requirement for the representationof the person-environment relation is that it supportthe derivation of appraisal variables. Moreover, aswe argue that appraisal is fast and uniform, the representationmust facilitate that assessment over the rangeof phenomena that induce emotional reactions.To address those requirements, EMA uses a representationbuilt on the causal representations developedfor decision-theoretic planning augmented by explicitrepresentation of intentions and beliefs (necessary forsocial attributions). The appraisal of relevance can be

expressed by a plan representation’s ability to expressthe causal relationship between events and states.These causal representations are also critical for assessingcausal attributions necessary for appraisingblame or credit-worthiness. Appraisal variables of desirabilityand likelihood can be modeled by the decision-theoreticconcepts of utility and probability. Explicitrepresentations of intentions and beliefs are alsocritical for properly reasoning about causal attributions,as these involve reasoning if the causal agentintended or foresaw the consequences of their actions.The commitments to beliefs and intentions also play arole in modeling coping strategies, especially what isoften called emotion-focused coping.We call the agent’s interpretation of its “agentenvironmentrelationship” the causal interpretation. Itprovides a uniform, explicit representation of theagent’s beliefs, desires, intentions, plans and probabilitiesthat allows uniform, fast appraisal processes,regardless of differences in the underlying phenomenabeing appraised. In the terminology of Smith andLazarus, the causal interpretation is a declarative representationof the current construal of the personenvironmentrelationship.Reactive and more deliberative processes map theirresults into this uniform representation. Architecturally,this is achieved by a blackboard-style model. Thecausal interpretation (corresponding to the agent’sworking memory) encodes the input, intermediate resultsand output of reasoning processes that mediatebetween the agent’s goals and its physical and socialenvironment (e.g., perception, planning, explanation,and natural language processing). At any point in time,the causal interpretation represents the agent’s currentview of the agent-environment relationship, whichchanges with further observation or inference. Wetreat appraisal as a set of feature detectors that mapfeatures of the causal interpretation into appraisalvariables. For example, an effect that threatens a desiredgoal is assessed as a potential undesirable event.Events are characterized in terms of appraisal variablesvia domain-independent functions that examinethe syntactic structure of the causal interpretation:• Perspective: viewpoint that the event judged• Desirability: what is the utility (positive or negative)of the event if it comes to pass, from the perspectivetaken (e.g., does it causally advance or inhibit a stateof some utility). The utility of a state may be intrinsic(agent X attributes utility Y to state Z) or derived(state Z is a precondition of a plan that, with somelikelihood, will achieve an end with intrinsic utility).• Likelihood: how probable is the outcome of theevent, derived from the decision-theoretic plan.• Causal attribution: who deserves credit/blame. Thisdepends on what agent was responsible for executingthe action, but may also involve considerations of intention,foreknowledge and coercion (see [10]).• Temporal status: is this past, present, or future• Controllability: can the outcome be altered by actionsunder control of the agent whose perspective istaken. This is derived by looking for actions in thecausal interpretation that could establish or blocksome effect, and that are under the control of theagent who’s perspective is being judged (i.e, agent Xcould execute the action).• Changeability: can the outcome be altered by someother causal agent.Each appraised event is mapped to an emotion instanceof a type and intensity following the structural schemeproposed by Ortony et al. [11]. A simple activation-basedfocus of attention model computes a current emotionalstate based on most recently accessed emotion instances.Another key aspect of EMA is that it includes acomputational model of coping integrated with theappraisal process (according to Lazarus’s theory).Coping determines how one responds to the appraisedsignificance of events. Coping strategies are proposedto maintain desirable or overturn undesirable in-focusemotion instances. Coping strategies essentially workin the reverse direction of appraisal, identifying theprecursors of emotion in the causal interpretation thatshould be maintained or altered (e.g., beliefs, desires,intentions and expectations). Strategies include:• Action: select an action for execution• Planning: form an intention to perform some act (theplanner uses intentions to drive its plan generation)• Seek instrumental support: ask someone that is incontrol of an outcome for help• Procrastination: wait for an external event to changethe current circumstances• Positive reinterpretation: increase utility of positiveside-effect of an act with a negative outcome• Acceptance: drop a threatened intention• Denial: lower the probability of a pending undesirableoutcome• Mental disengagement: lower utility of desired state• Shift blame: shift responsibility for an action towardsome other agent• Seek/suppress information: form positive or negativeintention to monitor a pending or unknown stateStrategies give input to the cognitive processes that actuallyexecute these directives. For example, planful copinggenerates an intention to act, which in turn leads to theplanning system to generate and execute a valid plan toaccomplish this act. Alternatively, coping strategiesmight abandon the goal, lower the goal’s importance, orre-assess who is to blame.Not every strategy applies to a stressor (e.g., an agentcannot be problem directed if it is unaware of actionsimpacting the situation), but multiple strategies can apply.EMA proposes strategies in parallel but adopts themsequentially. A set of preferences resolve ties: e.g., EMAprefers problem directed strategies if control is appraisedas high (take action, plan, seek information), procrastinationif changeability is high, and emotion-focus strategiesif control and changeability are low.

Figure 2: The causal interpretation at time t 4In developing a computational model of coping, wehave moved away from broad distinctions of problemfocusedand emotion-focused strategies. Formally representingcoping requires a crispness lacking from theproblem-focused/emotion-focused distinction. In particular,much of what counts as problem-focused copingin the clinical literature is really inner-directed in aemotion-focused sense. For example, one might forman intention to achieve a desired state – and feel betteras a consequence – without ever acting on the intention.Thus, by performing cognitive acts like planning,one can improve ones interpretation of circumstanceswithout actually changing the physical environment.To summarize, an agent’s causal interpretation isequated with the output and intermediate results ofprocesses that relate the agent to its physical and socialenvironment. This configuration of beliefs, desires,plans, and intentions represents the agent’s currentview of the agent-environment relationship, whichmay subsequently change with further observation orinference. We treat appraisal as a mapping from domain-independentfeatures of causal interpretation toindividual appraisal variables. Multiple appraisals areaggregated into an overall emotional state that influencesbehavior. Coping directs control signals to auxiliaryreasoning modules (i.e., planning, action selection,belief updates, etc.) to overturn or maintain featuresof the causal interpretation that lead to individualappraisals. For example, coping may resign the agentto a threat by abandoning the desire. The causal interpretationcould be viewed as a representation of workingmemory (for those familiar with psychologicaltheories) or as a blackboard (for those familiar withblackboard architectures).Model of the BirdThe bird example can be readily encoded into EMA.Some of the dynamics of the situation arise from theworld but others arise from the inferential processes ofthe model. Here we describe an encoding that producesthe hypothesized emotional transitions that we derivedfrom our video analysis.Our goal is not to definitively explainand reconstruct the inferencesand emotions experienced by this actor,but rather to illustrate how suchdynamic situations could be modeledby EMA. Many encodings are possible.We did, however, try to adoptwhat we felt were plausible inferencesand beliefs based on post hoc analysisof the situation.Figure 2 illustrates a snapshot ofEMA's causal interpretation severalsteps into the situation. EMA makesdiscrete changes to this interpretationover time, as a result of perceptual andinferential updates. (The Soar cognitivearchitecture, upon which EMA isbased, assumes that updates occur once every 100 milliseconds.)The time stamps at the bottom (e.g., t 0 )indicate the discrete time step in which elements areadded or deleted from the causal interpretation. Verticalrectangles indicate actions including two degenerateactions *init* (whose effects generate the initialstate of the simulation) and *goal* (whose preconditionscorrespond to the agent's goals). Ovals indicatepredicates that describe the current beliefs of somefeature of the world, including its likelihood of beingtrue and its utility. For example, "have Um" indicatesthat the actor has an umbrella (Um) and believes thiswith certainty (Pr=1.0) and that this fact has no intrinsicvalue (U=0.0). Predicates are linked to actions(representing the actions preconditions or effects) or toother states via establishment links (i.e., this effectestablishes a precondition for some other action) orthreat links (i.e., this effect deletes a precondition forsome other action). So, for example, the simulationbegan at time t 0 and the actor was healthy, holding anumbrella and the umbrella was lowered. Finally, 3Drectangles indicate appraisal frames. Each frame consistsof number, indicating at which time step it wasgenerated, a set of appraisal variables and an emotionallabel (due to space, the appraisal variables areomitted for all but the 4th frame.To complete a model, one must provide several additionalelements that sit outside the causal interpretation.EMA is provided with a plan library that consistsof a set action definitions and set of recipes indicatinghow these actions could be combined. These recipesare used both for plan generation and plan recognition(as when inferring behavior of other agents based onobserved actions and world states). EMA can also beprovided with a set of simple stimulus-response rulesthat automatically trigger actions when certain worldstate configurations are perceived. Finally, EMA mustbe connected to some world simulation that defineshow percepts change as a result of actions. We describethese additional elements in the course of describingthe evolving situation.

At time t 0 the actor has the high utility goal of beinghealthy and this goal has been established in the initialstate. This establishment is automatically appraised asdesirable and certain, resulting in joy. 1At time t 1 a sound is perceived, denoted in thecausal interpretation by the predicate "sound". As nocurrently executing action predicted that a soundwould occur, this is unexpected and EMA records thisunknown event in the causal history. EMA supportspartial observability of the world's state and this modelwe assume the bird cannot be seen unless the actorlooks at it. To model this, we define a S-R rule thatexecutes the “Look” action if a sound is perceived.At time t 2, EMA has perceived the flying bird. Typically,EMA is attached to a simulation environmentthat would maintain the world state and compute theobservable effects of actions such as "look." For themodel, we accomplish this through a domain-specificprocedure that sets "flying bird" to be true 100 ms afterlooking at the sound. By definition in the model,"flying bird" has some negative utility for the action(U=−10). Thus, the effect of this action is appraised asundesirable and certain, leading to distress.At time t 3, EMA infers the bird will attack. This followsfrom a general plan recognition approach informedby the domain-specific plan recipes. This actionhas one precondition "bird-flying" that is establishedby the unexpected event in t 1 . It has one effect"injured" denoting that the actor will become unhealthyas a result of this action and this threatens theactor's goal of being healthy. 2 This threat relation isautomatically appraised as undesirable, uncertain (asthe effect may not occur), and uncontrollable (as thereis no explicit way to respond to the threat, given thecurrent causal interpretation). The result is Fear.At time t 4, EMA infers that there is a way to confrontthe threat to the actor's health: whack the birdwith the umbrella. This follows from the general plangeneration approach informed by the domain-specificrecipes. The planner has determined that the bird's"plan" can be confronted by blocking its precondition.The planner further infers that the probability of thebird's "plan" succeeding is significantly reduced.Given that there is now an action in the causal interpretationunder control of the actor and addressing thethreat to health, the threat to health is automaticallyreappraised as controllable. The result is Anger.We have modeled the scenario to this point, thoughthe remainder is straightforward. The model must nextinfer that it must raise the umbrella to satisfy the"whack" plan and subsequently execute this initial1 Arguably, one has a no emotions to being healthy but onlyreacts when there is a threat to health or they are unhealthy.EMA allows a utility distribution over predicates but we omitthis distinction in our example for simplicity.2 In actuality, the effect of this action is to make “healthy”false (i.e., “injured” is shorthand for NOT(healthy)). The sameholds for “dead bird” which denotes NOT(flying).plan step. This plan, however, is overtaken by eventsas the bird becomes caught in another actor's hair, disablingthe "attack" and possibly resulting in the undesirablestate that the bird will be injured.Concluding RemarksComputational models of psychological phenomena arepowerful research tools. The development of a model canhelp concretize theories, reveal shortcomings and canhelp derive predictions through simulations.EMA provides a framework for exploring and explainingemotion dynamics. In particular, the simulationof the bird example, and the emotional dynamicsit reveals, argues that the temporal characteristics ofappraisal may be a by-product of other perceptual andcognitive processes. By modeling appraisal as a fast,uniform processes, EMA roots the temporal dynamicsin those other processes that operate on the causal interpretation.Further, EMA’s description of appraisal iseconomical, not requiring appeal to alternative fast andslow appraisal processes.AcknowledgmentsThis work was supported by the HUMAINE NetworkReferences1. Lazarus, R., Emotion and Adaptation. 1991, NY: OxfordUniversity Press.2. Reisenzein, R., Appraisal Processes Conceptualized from aSchema-Theoretic Perspective, in Appraisal Processes inEmotion: Theory, Methods, Research, K.R. Scherer, A.Schorr, and T. Johnstone, Editors. 2001, Oxford Press. p..3. Scherer, K.R., Appraisal Considered as a Process of MultilevelSequential Checking, in Appraisal Processes in Emotion:Theory, Methods, Research, K.R. Scherer, A. Schorr,and T. Johnstone, Editors. 2001, Oxford University Press..4. Smith, C.A. and L. Kirby, Consequences require antecedents:Toward a process model of emotion elicitation, inFeeling and Thinking: The role of affect in social cognition,J.P. Forgas, Editor. 2000, Cambridge University Press.5. Moors, A., et al., Unintentional processing of motivationalvalence. The Quarterly Journal of Exp. Psychology, 2005.6. Smith, C.A. and R. Lazarus, Emotion and Adaptation, inHandbook of Personality: theory & research, L.A. Pervin,Editor. 1990, Guilford Press: NY. p. 609-637.7. Perrez, M. and M. Reicherts, Stress, Coping, and Health.1992, Seattle, WA: Hogrefe and Huber Publishers.8. Gratch, J. and S. Marsella, Evaluating a computationalmodel of emotion. Journal of Autonomous Agents and MultiagentSystems (in press), 2005.9. Gratch, J. and S. Marsella, A domain independent frameworkfor modeling emotion. Journal of Cognitive SystemsResearch, 2004. 5(4): p. 269-306.10. Mao, W. and J. Gratch. Social Causality and Responsibility:Modeling and Evaluation. in International Working Conferenceon Intelligent Virtual Agents. 2005. Kos, Greece.11. Ortony, A., G. Clore, and A. Collins, The Cognitive Structureof Emotions. 1988: Cambridge University Press.