Human–Robot Interaction - LASA - EPFL

© CORBIS & JOHN FOXX
An Exclusive Course for
Computer Scientists and Engineers
The field of human–robot interaction (HRI)
addresses the design, understanding, and evaluation
of robotic systems, which involve humans and
robots interacting through communication [1]. As
the field matures, education of students
becomes increasingly important.
Courses in HRI provide the canonical
set of knowledge and core skills
that represent the current state of the
field and permit the evolution of
knowledge and methods to be transferred
from research to a broad set of
students. In addition, coursework in HRI
creates a workforce capable of transferring
Digital Object Identifier 10.1109/MRA.2010.936953
BY ROBIN R. MURPHY,
TATSUYA NOMURA,
AUDE BILLARD,
AND JENNIFER L. BURKE
HRI theory to practice. However, as would be expected with an
emerging field, HRI courses are largely ad hoc.
Teaching HRI is challenging because the subject is multidisciplinary,
and there is lack of educational materials, such as textbooks
and resources such as robots and interfaces.
This article summarizes the discussion and
findings from the “Teaching Humans
About Human–Robot Interaction”
workshop on the development of an
HRIcourseforcomputerscientistsand
engineers. This half-day workshop was
held at the IEEE/Robotics Society of
Japan International Conference on Intelligent
Robots and Systems (IROS), 22 September
2008, in Nice, France. The motivation for the workshop was a
direct response to a key finding from the National Science
JUNE 2010 1070-9932/10/$26.00ª2010 IEEE IEEE Robotics & Automation Magazine 85

The IEEE Robotics and Automation
Society sponsors a Technical
Committee on Human–
Robot Interaction.
Foundation (NSF)-sponsored HRI Young Pioneers Workshops
[2], held in conjunction with the annual Association of Computing
Machinery (ACM)/IEEE Conference on Human–Robot
Interaction. The findings consistently emphasized the need for
an interdisciplinary course or curriculum in HRI to be taught at
the university level. However, until the IROS workshop, there
has been no reported venue for faculty to gather and discuss such
a curriculum or teaching methods. Although this workshop was
limited in both time and the number of participants, it offers a
starting point and some insight into HRI education.
The objectives of the workshop were to identify what is
essential in an HRI course by leveraging the experiences to
date in teaching HRI and then use this list of fundamentals to
define course content. The workshop was also expected to
create a community of educators within the emerging HRI
research community, foster the exchange of best practices and
pedagogical methods, and provide reference materials, if any,
for instructors teaching HRI.
The rest of this article is organized as follows. It first
describes the workshop in terms of participants and activities.
The challenges for a course in HRI identified by the participants
follow next. Suggested course content, both in terms of
the set of candidate topics for a course, and a sequence of lectures
for advanced students in artificial intelligence (AI) and
robotics follows. Next, possible course projects and assignments
are discussed. The article then concludes with a distillation of
the workshop into a set of six major findings.
Workshop Description
The workshop was attended by 18 participants, representing
France, Germany, India, Japan, Korea, Switzerland, and the
United States, and was organized around group discussions.
Graduate students slightly exceeded the number of professors
and industry researchers. The workshop consisted of four
parts; beginning with each participant positing what they
believed should be included in a HRI course and what is currently
missing from HRI education.
Second, a discussion on available resources for HRI education
was initiated with an invited talk by Dr. Kojiro Matsushita
from the University of Tokyo and demonstration of his two
low-cost robot kits [3]. One kit was made from servomotors
and plastic water bottles and can be constructed by beginners
in less than 6 h after 2 h of instruction, making it suitable for
nonrobotics students. The resulting robot can take many configurations,
including legs and snake structures, and more
emotive shapes similar to puppets. The robot can be controlled
directly, learn motions from the user guiding the robot, or
controlled by noninvasive contact sensors measuring muscle
strain. Another kit is based on a toy hand. Dr. Matsushita is
working on an English translation of his book on how to build
and use these robots.
The next discussion, led by Dr. Aude Billard from the
Ecole Polytechnique Federale de Lausanne (EPFL), concentrated
on lessons learned from both instructor and student
experiences with HRI. Three professors described HRI classes
at the Indian Institute of Information Technology Allahabad
(Prof. G.C. Nandi), EPFL (Prof. Aude Billard), and at the
University of South and Texas A&M (Prof. Robin Murphy).
Rod Gutierrez, a graduate student at the University of South
Florida, presented feedback from the 2008 Young Pioneers
Workshop at the 2008 ACM/IEEE International Conference
on Human–Robot Interaction with amplifying comments
from that workshops attendees.
The fourth discussion took the form of breakout groups.
Participants were split into two groups: one to discuss the perfect
syllabus, or sequence of lectures, for an HRI course, and
the other to determine the perfect set of assignments and projects
(Figure 1). The groups then gave reports summarizing
their thoughts. The fifth component was a short recap and
discussion of future activities, primarily increased involvement
in the annual HRI conference.
Figure 1. Participants in one of the two breakout groups.
(Photo courtesy of Robin Murphy.)
Challenges
Creating a new course is always challenging, but the field of
HRI provides three additional challenges for education.
First, HRI is multidisciplinary, incorporating contributions
from communications, computer science, engineering, psychology,
and theater, creating challenges in creating course
content that covers the field in sufficient depth without requiring
a large number of prerequisites. Balancing coverage depth
while minimizing prerequisites is particularly hard because the
background between the engineering sciences and the human
sciences was felt to be large. In response, the workshop participants
quickly restricted discussion to teaching students in the
engineering sciences; even within that restriction, the differences
between individual engineering disciplines and computer
science were significant.
86 IEEE Robotics & Automation Magazine
JUNE 2010

Second, the diversity of the HRI field also extends to
resources, and as a result, there are no dedicated HRI resources,
although possible materials can be extracted from mature
fields. For example, HRI does not have a journal or textbook.
There is a dedicated conference, the annual ACM/
IEEE International Conference on HRI now in its fourth
year, but the majority of participants were not aware of the
conference. A related conference, the IEEE International
Symposium on Robot and Human Interactive Communication
(Ro-Man), also publishes HRI research. The IEEE
Robotics and Automation Society sponsors a technical committee
on HRI.
Third, there is a lack of cost-effective, pedagogically
appropriate robots and rich interfaces. As detailed below,
hands-on projects in HRI are highly desirable. Robots such
as Lego Mindstorms are inexpensive and do not require
extensive programming expertise but, as noted by the students,
may not provide sufficient capability to support key
HRI topics. Humanoid robots vary in price but often have
significant limitations for general HRI topics. For example,
the design of the HOAP-3 robot prevents the camera in the
head from seeing the hands, curtailing physical interaction
and learning manipulation tasks. Few robots in any price
range support human–computer interfaces such as haptics,
touch screens, or gestures. Speech recognition remains unreliable,
obviating the easy application of natural language to a
survey course. One promising robot resource that addresses
the third challenge is the robot kits presented by Dr. Matsushita
and shown in Figure 2.
Course Content
Starting from the discussion and
throughout the workshop, topics for
inclusion in course emerged. The objectives
of an HRI course were proposed,
and a subset of these topics was arranged
into one possible sequence of lectures
aimed at advanced robotics or AI students.
The individual topics were not
rated as to relative importance because
of time constraints.
The topics not only largely borrowed
from robotics, AI, and psychology themes
but also included more unique HRI subjects
and applications. Robot control and
humanoid robot design and control were
two robotcentric topics suggested for
inclusion, along with user interfaces. Skill
acquisition, often associated with traditional
robot learning, has been experiencing
a renaissance with the new emphasis
offered by HRI. In particular, it was
noted that students often do not understand
limits on the range of motion or
degrees of freedom in humanoid robots
and thus become confused when trying
to generate naturalistic motions. Natural
(a)
Robot control and humanoid robot
design and control were two
robotcentric topics suggested for
inclusion.
language processing and machine learning, staples of AI, were
also deemed important. Psychology and cognitive engineering
topics were tools and methods to measure HRI, joint attention
theory, teams, and user-centered design. The participants noted
that there was no concise list of qualitative and quantitative evaluation
methods or tools, nor was there a clear mapping of particular
techniques to desired outcomes, e.g., what technique
would be best to measure X Social behaviors, emotion or affective
expressions, interaction modalities, social learning, user
expectations, safety, the Uncanny Valley, and ethics emerged as
unique HRI topics. The topic of social behaviors actually is
composed of two topics: one is “what are social behaviors” and
the other is “how can robots be programmed to generate social
behaviors” Rehabilitation and therapy was singled out as major
HRI application areas.
Course objectives should include at a minimum:
u definition of HRI
u the basic modalities for interacting with a robot
u the key issues in HRI
u the current applications
u the process of making robots into social platforms
u the importance of social skills in robots (role of learning,
a theory of mind).
Figure 2. Demonstration of walking robot and robot hand kits by Dr. Matsushita. (a)
Legged robot from water bottles and (b) robot hand. (Photo courtesy of Rodrigo
Gutierrez.)
(b)
JUNE 2010 IEEE Robotics & Automation Magazine 87

Psychology and cognitive
engineering topics were tools and
methods to measure HRI, joint
attention theory, teams, and usercentered
design.
The prerequisites for an HRI course depend on the target
audience and scope of material, although probability and statistics
was considered a universal prerequisite. In addition to
probability and statistics, related concepts such as regression
analysis and experimental design would be helpful for a
course focused on methodology. Robotics and AI (capturing
control and automation), sensors, and machine vision
are starting points for robotics students to study HRI. In
addition, having signal processing and machine learning
might be very helpful, although participants noted that
machine learning was a topic that should also be covered in
the course.
Assuming an advanced robotics student with a background
in AI, a set of possible lectures spans robot inputs to ethics.
These are:
u modalities and types of knowledge acquired through
interactions, including vision, speech, and haptics
u representing the world and the intentions of others
u case studies of social learning and interaction
u evaluation methodologies, both qualitative and quantitative
u ethics.
Course Projects and Assignments
Having a hands-on component to an HRI class was strongly
recommended by those who have taught HRI, who wish to
teach HRI, and students. The recommended pattern was to
have a series of small assignments either directly related to
the current course material or scaffolded in complexity, then
a final project chosen by the students. Assignments and projects
directly involving robots and users were seen as the most
desirable. However, working with users and robots raises
many issues. Availability of platforms and of users is a concern.
User-studies often requires a great deal of planning and
preparation, including getting any institutional human–subject
protocol approvals. Working with robots is costly, and
there are concerns that sufficient robots will not be operational
when needed. Robot simulations may prove to be
a viable alternative to directly using a robot. Simulations
such as Microsoft Robotics Studio can be programmed at a
high level of abstraction, allowing the students to move and
direct the robot without having to focus on the details of the
robot or robot programming. Regardless of whether real or
simulated robots are used, two applications are particularly
attractive for a course. Search and rescue robotics has a
strong societal benefit, whereas social robots are engaging
and entertaining.
Summary of Findings
The workshop focused on teaching roboticists (computer science
and engineers) at graduate level, generally discussing
issues from an instructor’s viewpoint (e.g., pedagogy and
resources) with a presentation and feedback from students.
The six findings from the workshop are summarized below.
u Finding 1: Students prefer HRI courses with a high
degree of interaction between students and between
students and robots over courses that are primarily lecture
based. Interaction, both through discussion and
hands-on projects, appears to be the desired style for
teaching HRI.
u Finding 2: Candidate topics for coverage in an HRI
course include emotion, ethics, humanoid robot design
and control, interaction modalities, joint attention theory,
machine learning, natural language processing, robot
control, safety, skill acquisition, social behaviors, social
learning, teams, tools and methods to measure HRI, the
Uncanny Valley, user interfaces, user-centered design,
and user expectations. The choice of topics to include
depends on the course prerequisites. On one hand, course
prerequisites permit content to go deeper or free up time
in the course schedule to include more of these topics.
On the other hand, prerequisites may exclude students
from the human sciences or even from a particular engineering
science discipline. This could undermine the
benefits of interdisciplinary courses and the discussionoriented
teaching style desired by the students.
u Finding 3: The most prominent deficits for creating
course content in HRI are the lack of: 1) a set of key
principles of HRI, 2) a survey of mechanisms on how to
generate social behaviors, and 3) a succinct synopsis of
user evaluation methods. We note that the fist deficit in
the list reflects the lack of consensus in the HRI
community over HRI. However, the second and third
deficits highlight gaps in robotics that must be filled by
multidisciplinary work; the second deficit shows the
need to connect control theory with the behavioral sciences,
whereas the third deficit necessitates a transfer of
quantitative and qualitative methods pioneered outside
of robotics.
u Finding 4: The major missing pedagogical tools for
instructors are cost-effective robots and a corpus of case
studies, illustrating key principles of HRI. Cost is
viewed as a major driver of a robot that can be adopted
by a large number of universities for teaching HRI.
u Finding 5: Course development should consider industry
needs as well as instructor constraints and student learning
preferences, as not all students will become HRI
researchers. This includes understanding anthropomorphic
robots as well as nonanthropomorphic forms.
u Finding 6: Regardless of the target audience, an
HRI course will most likely require students to have
a background in statistics and will, at a minimum,
cover interaction modalities, issues, social interactions,
and applications.
88 IEEE Robotics & Automation Magazine
JUNE 2010

The workshop briefly touched on the way ahead. In terms
of facilitating general progress in HRI education, there was a
hope that the HRI conference would become a clearing house
for HRI-specific resources. In terms of continuing the discussion
on HRI education, it would be interesting to elicit the
viewpoints of other disciplines, especially psychology, on
what they believe are fundamental topics and how HRI
should be taught.
Acknowledgments
The authors thank Dr. Matsushita for his demonstration of
low-cost robots, Rod Gutierrez for his presentation and general
assistance during the workshop, Dr. Ephriam Glinert for
his support of the HRI Young Pioneers Workshop (NSF
Grant IS-0813909), and the IROS 2008 tutorial chairs, Dr.
Rachid Alami and Dr. Roland Siegwart.
Keywords
Human–robot interaction, robotics education.
References
[1] M. A. Goodrich and A. C. Schultz, “Human-robot interaction: A survey,”
Found. Trends Hum.-Comput. Interact., vol. 1, no. 3, pp. 203–275, 2007.
[2] J. Burke, R. Murphy, and C. Kidd, “Young researchers in HRI workshop
2006,” Interact. Stud., vol. 8, no. 2, pp. 343–358, 2007.
[3] K. Matsushita, H. Yokoi, and T. Arai, “Plastic-bottle-based robots in educational
robotics courses—Understanding embodied artificial intelligence,”
J. Robot. Mechatron., vol. 19, no. 2, pp. 212–222, 2007.
Robin R. Murphy received a B.M.E. degree in mechanical
engineering, and M.S. and Ph.D. degrees in computer science in
1980, 1989, and 1992, respectively, from Georgia Tech, where
she was a Rockwell International Doctoral Fellow. She is the
Raytheon Professor of Computer Science and Engineering at
Texas A&M. In 2008, she was awarded the Al Aube Outstanding
Contributor Award by the Association for Unmanned Vehicle
Systems International Foundation for her insertion of ground,
air, and sea robots for urban search and rescue at the 9/11 World
Trade Center disaster, Hurricanes Katrina and Charley, and the
Crandall Canyon Utah mine collapse. She is a distinguished
speaker for the IEEE Robotics and Automation Society and has
served on numerous boards, including the Defense Science
Board, U.S. Air Force Scientific Advisory Board, NSF Computer
and Information Science and Engineering Advisory Council,
and the Defense Advanced Research Projects Agency Information
Science and Technology Study Group. She is a Senior
Member of the IEEE. Her research interests include AI, HRI,
and heterogeneous teams of robots.
Tatsuya Nomura received the M.S. degree in mathematics
from Osaka University, Japan, in 1989, and the D.E. degree in
engineering from Kyoto University, Japan, in 1998. From 1989 to
2000, he was with the Corporate Research and Development
Group at Sharp Corporation. From 2000 to 2004, he was with
Hannan University, Osaka, Japan. He is currently an associate
professor in the Department of Media Informatics, Ryukoku
University, Otsu, Japan, and a researcher in the Advanced
The prerequisites for an HRI
course depend on the
target audience and scope
of material, although probability
and statistics was considered a
universal prerequisite.
Technology and Research Intelligent Robotics and Communication
Laboratories, Japan. He is a member of the Japanese
Psychological Association, the Japanese Cognitive Science
Society, and the Mathematical Society of Japan. He is a Member
of the IEEE. His research interests include intelligent
robots and human—robot interaction.
Aude Billard received a B.Sc. degree in physics from EPFL,
with specialization in particle physics at the European Center for
Nuclear Research (CERN) in 1994. She received her M.Sc.
degrees in physics from the same university, with specialization in
particle physics at the CERN and in knowledge-based systems in
1996 and a Ph.D. degree in AI from the Department of Artificial
Intelligence at the University of Edinburgh in 1998. She is an
associate professor and head of the Learning Algorithms and Systems
Laboratory at the School of Engineering, EPFL. Before this,
she was a research assistant professor at the Department of
Computer Sciences at the University of Southern California,
where she retained an adjunct faculty position to this day. She is a
Member of the IEEE. Her research interests focus on machine
learning tools to support robot learning through human guidance.
This extends also to research on complementary topics,
including machine vision and its use in human–machine interaction
and computational neuroscience to develop models of
learning in humans.
Jennifer L. Burke received the B.A. degree in business from
Florida State University, the M.S. degree in counseling from
the University of North Florida, and the M.S. and Ph.D.
degrees in industrial-organizational psychology (minor: man–
machine interaction) from the University of South Florida, in
1980, 1990, and 2006, respectively. She is a practicing human
factors engineer at SA Technologies, specializing in robotic
interface design. She is active in the robotics and psychology/
human factors communities and is the author of more than 30
publications in fields of robotics, human performance, and
workplace studies. She is a member of the ACM, the American
Psychological Society, and the Human Factors and Ergonomics
Society. Her research interests include team processes
and human–robot interaction.
Address for Correspondence: Robin R. Murphy, Computer
Science and Engineering, Texas A&M University, College
Station, TX, USA. E-mail: murphy@cs.tamu.edu.
JUNE 2010 IEEE Robotics & Automation Magazine 89