Automotive User Interfaces and Interactive Vehicular Applications

Natural, Intuitive Hand Gestures -
A Substitute for Traditional DV-Control?
Research approach
� Driving and ADAS/IVIS operation is a complex visual-auditory task; overload/distraction affects driving safety
→ novel solutions for future vehicular interfaces are required to keep the driver's workload low
� Motion controllers (Wii Remote, PlayStation Move, Kinect) have revolutionized interaction with computer (games)
→ replacing button/stick based interaction with natural interaction based on intuitive gestures and body postures
� Mapping is „easy“ for games
...but is a highly competitive task for application in in-car UI‘s
→ body parts to be used? (hand, arm, head, back), region in the car (wheel, controls, gearshift), what are intuitive poses/
gestures?, which application(s) to control?, mapping command ↔ gesture?
Gestural interaction with an E-Mail client
� Predefinitions
- Driving in Europe (cultural assignment; manual transmission)
- Highway situation (free workload to process tasks; > 85% drive
gearshift
single handed on the highway (opinion survey) → gearshift area)
door mat
- Technique/sensory device (RGBD images → Kinect)
- Application to control: Apply participatory design
(step1; ask business traveler) → most votes for an E-mail client
(step2; ask freq. E-mail users) → user defined mapping of
hand poses/gestures
� User study
- Visual request to perform certain pose/gesture (in-car screen)
- Recognition rates of 0.98 for trigger task, ~0.90 for static poses, ~0.67 for dynamic gestures;
- Only 10% deterioration under dual task condition (auditory stimulus, cognitive/vocal response load as primary task)
Selection of 1) static hand poses/gestures 2) dynamic hand gestures (sequence of 5 images)
Gesture 1 „system on/off“ Gesture 2 „pause/resume“ Gesture 3 „mark email“
Gesture 4 „next email“
Initial findings and issues to work on
� Volunteers liked this natural and intuitive type of interface and would like to see it in future vehicles
� Further research will address
(i) Comparison of the gestural interface with conventional UI‘s based on button-/knob-control or speech input
(ii) Replication of the tests under (more) real workload conditions, e. g., in a on-the-road driving scenario
(iii) Customizable gestures (based on driver's personal preference) to achieve higher recognition rates, lower detection times
Contact
Andreas Riener, Michael Roßbory
JKU, Institute for Pervasive Computing, Altenberger Str. 69, A-4040 Linz, Austria
E-Mail: , Phone: +43 / 732 / 2468 - 1432
steering wheel
trigger gesture
Institute for
Pervasive Computing
Technology for People
front passenger seat
driver‘s arm