TESIS DOCTORAL - Robotics Lab - Universidad Carlos III de Madrid

80 Chapter 4. The Decision Making SystemLEARNING PROCESSHappinessSadnessWellbeingDrivesMotivationsself-reinforcementState-ActionEvaluationROBOT'SCONTROLARCHITECTUREFigure 4.5: The role of happiness and sadness in the DMSto obtaining rewards and avoiding punishments. Following this point of view, in this proposedDMS, happiness and sadness are used as the positive and negative reinforcementfunctions during the learning process respectively. Moreover, this approach seems consistentwith the drive reduction theory introduced in Section 2.3.1 where, according to thistheory, the drive reduction is the chief mechanism of reward.According to Starzyk [178], this learning process is a kind of Motivated Learning becauseit uses internal reward signals which are related to abstract motivations and goals(happiness and sadness are related with the robot’s wellbeing which refers to the robot’sdrives). Then, the actions are evaluated considering how well these actions satisfy theinternal goals.The appraisal processIn order to define happiness and sadness, the definition of emotion given by Ortony [179] istaken into account. In his opinion, emotions occur due to an appraised reaction (positive ornegative) to events. According to this point of view, in [83], Ortony proposes that happinessoccurs because something good happens to the agent. On the contrary, sadness appearswhen something bad happens. Moreover, according to Esau [6], the satisfaction of a driveis usually accompanied by positive emotions reflecting the associated comfortable feeling.On the contrary, if a drive is not satisfied over a longer period of time, negative emotionscan often arise. These positive and negative emotions can be related to the happiness andsadness emotions.In the proposed system, happiness is related to a reduction of a drive (e.g., a positivereaction because the robot recharges its battery) and sadness to an increment of a drive(e.g. a negative reaction because the robot was damaged by a user). Taking into account