NUI Galway – UL Alliance First Annual ENGINEERING AND - ARAN ...

Non Verbal Communication within Collaborative Virtual Environments
Alan Murphy and Dr. Sam Redfern
Department of Information Technology
National University of Ireland Galway
a.murphy30@nuigalway.ie, sam.redfern@nuigalway.ie
Abstract
This research aims to examine the usefulness of
achieving unobtrusive avatar control in Collaborative
Virtual Environments (CVE’s) as a means of tracking
and communicating non verbal cues. Through the
capture of user data in real time their avatar can in
turn be animated to replicate user movement in real
time. This passive control of the avatar will provide for
a more realistic representation of a user in the virtual
world, which in turn will provide a channel through
which non verbal clues can be communicated and
interpreted by participating users of the CVE.
1. Introduction
Collaborative Virtual Environments are distributed
virtual reality systems with multi user access. There are
many potential applications of these groupware systems
ranging from learning environments to remote
conferencing or simply virtual business meetings.
In everyday face to face interactions, participants are
able to utilize a full range on non-verbal
communicational resources. These resources include
the ability to move their head to look at each other,
point or use hand gestures to address objects, change
their gaze direction, posture, facial expression or their
position [1].
Social psychologists have argued that more than
65% of the communicational information exchanged
during such a face to face encounter is carried on the
non verbal band [2]. Therefore, it is given that there is a
need to provide support for such channels of
communication when designing a platform for remote
person to person communication.
2. Previous Work
Much work has been done in the areas of presence,
immersion and awareness in CVE’s, all pertinent topics
with regard to rating the quality of interactions in a
virtual setting. To date many approaches taken to
capture user data to recognize a hand gestures have
involved mountable sensors which are used for
mapping a hand movement of to that of an avatar [3]. In
terms of capturing a participants emotional state much
research has been based on capturing the fundamental
human emotions as described in the works of emotional
psychologists like Robert Plutchnik [4] and Paul Ekman
[5], who propose separate but similar lists of 6-8
fundamental human emotions. Most prototypic
development of emotion capture solutions as applied to
CVE’s to date, have solely been trained to capture this
12
static & finite list of primary emotions, lacking the
dynamic capacity of capturing other unread emotions.
3. Proposed Work
The proposed work will investigate the usefulness of
a user passively controlling their avatar’s facial
expression by mapping their own expressions onto their
avatar in real time, to convey further unread emotional
states. Work to date has limited itself to focus on a few
trained fundamental emotions, whereas in reality there
are many more subtle emotions that only facial
expressions capture. By adopting the more dynamic
approach of monitoring all expressions instead of
searching for a trained few, the non verbal channel of
communication could be fully utilized thus improving
user communication and sense of presence. A sample
experiment may involve giving subject A tasks or
readings to trigger these untrained emotions, and have
subject B monitor the expressions of A’s avatar keeping
a log of interpreted emotions. Consultation between
both subjects would then determine the accuracy of B’s
perceptions. The technique of tracking user’s facial data
from a live video stream [Fig 1] and using this data to
control an avatar’s expression in real time could offer a
dynamic solution for tracking emotional states beyond
Plutchnik and Ekman’s fundamental taxonomies.
Fig 1. Extracting user’s head orientation from video stream.
4. References
[1] Steptoe, W., Wolff, R., “Eye Tracking for Avatar
Eye-Gaze and Interactional Analysis in Immersive
CVE’s”, CSCW’08, ACM, USA, 2008, pg 197.
[2] Fabri, M., Moore, D.J., Hobbs, D.J., Smith, A.B.,
“The Emotional Avatar: Non-verbal Communication
between Inhabitants of CVEs”, GW’99, Springer-
Verlag, Berlin, 1999, pg 269.
[3] Broll,W. Meier, E., Schardt, T. “Symbolic Avatar
Acting in Shared Virtual Environments”, Workshop on
Behavior Planning for Life-Like Characters and
Avatars, Sitges, Spain, 1999
[4] Plutchik, R., “The Emotions: Facts, Theories, and a
New Model”, New York: Random House. 1962