Automotive User Interfaces and Interactive Vehicular Applications

ABSTRACT
Natural and Intuitive Hand Gestures:
A Substitute for Traditional Vehicle Control?
A. Riener and M. Rossbory
Institute for Pervasive Computing, Johannes Kepler University, Linz/Austria
E-Mail: riener@pervasive.jku.at, Tel. +43(0)732-2468-1432
This poster aims at discussing the potential of natural, intuitive
hand poses and gestures used for interaction with standard
applications in a vehicle while driving. Test drivers
liked this natural type of interface and indicated that invehicle
function control with intuitive hand gestures is a
promising approach, which they would like to see in real
cars in the future.
1. INTERACTION WITH THE VEHICLE
Driving, and in particular interaction with and control of
information and assistance systems in a car, gets more and
more complex because of the increasing number of buttons,
switches, knobs, or the multi-stage functionality of controls
such as BMW iDrive or Audi MMI. A possible consequence
from the excessive information is cognitive overload, affecting
driving performance, and finally resulting in driver distraction.
This complexity-overload relation emphasizes the
importance of novel solutions for future vehicular interfaces
to keep the driver’s workload low. As the visual and auditory
channels are mainly responsible for driving and driving
related tasks, interaction modalities discharging these channels
bear good prospects to reach this objective. On the
other hand, however, care must be taken that driving safety
is not affected, e. g., due to a violation of the “eyes on the
road, hands on the wheel” paradigm.
1.1 Gestural interaction in vehicular UI’s
With emergence of motion controllers such as Wii Remote/Nunchuk,
PlayStation Move, or Microsoft Kinect, interaction
with computer games was revolutionized, as, for
the first time, button/stick based controllers previously used
were replaced with natural interaction based on intuitive
gestures and body postures inspired from reality. The use
of gestural interfaces carries also lot of potential for (driving
unrelated) control tasks in vehicles, supported by recent
work reporting fewer driving errors when using gestural interfaces
in the car as compared to common “physical” interfaces
[1]. In this work, we focus on gestural interaction
and use it for application control in the car. Static hand
poses/dynamic hand gestures are (i) performed while driving,
(ii) recorded with a RGBD depth camera (Kinect), (iii)
evaluated with a recognition framework, and finally (iv) reviewed
systematically.
For the initial setting, predefinitions were made to use
Copyright held by author(s).
AutomotiveUI’11, November 29-December 2, 2011, Salzburg, Austria.
Adjunct Proceedings
- 33 -
the interface on the European market and in highway driving
only. This allows the gesture set to be restricted to
the interaction behavior of Europeans (to address the issue
of cultural differences) and, as most European cars have a
manual shift, to use the gearshift area for gathering driver
gestures. Furthermore, we clearly accentuate that this setting
does not compromise the“hands on the wheel”paradigm
as (i) only secondary functions are controlled, thus timing is
not an issue and interaction is not necessary at all, (ii) the
workload of a driver is lowest on the highway [4], and (iii)
from a opinion survey it came up that less than 15% out of
>150 respondents drive with both hands on the wheel while
on the highway.
1.1.1 Considerations on the gesture set
While the definition of gestures to control computer games
is often straightforward and mostly borrowed from the real
world, e. g., a player of a tennis game would immediately
know about how to use the controller without learning, this
is in general not the case for “artificial” interaction as dominant
in the vehicle. Interface designers often define gestures
on its own preferences, evaluate them in user studies, apply
modifications and finally teach the users on how to employ
a gesture to interact with the system in the desired manner.
This is, of course, not the optimal way, as users may
have different personal preferences of how they would like to
interact with the system to perform a certain task. The predefinition
of gestures independent from a driver’s personal
bias is indeed not the preferred way – in contrast, it is essential
that gesture based interaction is observed as natural as
possible in order to keep driver’s mental workload low (that
is, to avoid that he/she has to think about which gesture to
use for what action or how a specific gesture is defined).
A
1 „Activate“ (on)
2 „Deactivate“ (off)
B
3 „Pause“
4 „Resume“
C
5 „Mark“
D
6 „Delete“ (throw
over the shoulder)
E
7 „Next Email“
8 „Prev. Email“
F
9 „Next Day“
10 „Prev. Day“
Figure 1: Static hand poses (left)/dynamic gestures
(right) and mapping to 10 application commands.
By incorporating the user in the design phase of such an
interface it could be defined and parametrized in the optimal
way. The participatory design in the actual case came
up with an Email client as the application of choice, with 6
static poses/dynamic gestures to control 10 application functions
(Figure 1). The static pose indicated as “A” is the so-