TESIS DOCTORAL - Robotics Lab - Universidad Carlos III de Madrid

The robot decides its future actions based on what it has learned from previous experiences.Although the current context of this robot is limited to a laboratory, the socialrobot cohabits with humans in a potentially non-deterministic environment. The robot isendowed with a repertory of actions but, initially, it does not know what action to executeeither when to do it. Actually, it has to learn the policy of behavior, i.e. what action toexecute in different world configuration, that is, in every state, in order to satisfy the driverelated to the highest motivation. Since the robot will be learning in a real environment interactingwith several objects, it is desired to achieve the policy of behavior in an acceptablerange of time.The learning process is performed using a variation of the well-known Q-Learningalgorithm, the Object Q-Learning. By using this algorithm, the robot learns the value ofevery state-action pair through its interaction with the environment. This means, it learnsthe value that every action has in every possible state; the higher the value, the better theaction is in that state. At the beginning of the learning process these values, called the Qvalues, can all be set to the same value, or some of them can be fixed to another value.In the first case, this implies that the robot will learn from scratch; in the second case, therobot has some kind of previous information about the action selection. These values areupdated during the learning process.The emotion of fear is particularly studied. The generation process of this emotion (theappraisal) and the reactions to fear are really useful to endow the robot with an adaptivereliable mechanism of “survival”. This dissertation presents a social robot which benefitsfrom a particular learning process of new releasers of fear, i.e. the capacity to identify newdangerous situations. In addition, by means of the decision making system, the robot learnsdifferent reactions to prevent danger according to different unpredictable events. In fact,these reactions to fear are quite similar to the fear reactions observed in nature.Another challenge is to design a solution for the decision making system in such a waythat it is flexible enough to easily change the configuration or even apply it to differentrobots.Considering the bio-inspiration of this work, this research (and other related works) wasborn as a try to better understand the brain processes. It is the author’s hope that it shedssome light in the study of mental processes, in particular those which may lead to mentalor cognitive disorders.vi