Slides - Institute for Computer Graphics and Vision

icg.tugraz.at

Slides - Institute for Computer Graphics and Vision

Snap-to-Feature Interface for

Annotation in Mobile

Augmented Reality

Gun A. Lee, Ungyeon Yang, Yongwan Kim, Dongsik Jo, Ki-Hong Kim

Electronics and Telecommunications Research Institute

(ETRI)

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

1


Introduction

Advances in mobile computing platforms

Even VR and AR applications are available

Wikitude on Google Phone (2008) Bionic-Eye on iPhone (2009)

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

2


Introduction

Touch interfaces getting popular in mobile computing &

AR platforms

Accurate touch interaction is not that easy in mobile

environments

Situation gets worse with handheld mobile AR

Display surface & camera viewpoint moves

How to overcome this problem?

Zooming (Loupe in Apple iPhone)

Freezing (Güven et al. 2006, Lee et al. 2009)

Snapping?

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

3


Snapping

A well known interaction method in GUI

The interaction point (i.e., pointer or cursor)

attracted to graphical features (e.g., grids, vertices, lines)

when it gets close enough

In AR environment

graphical features => features of the physical scene

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

4


Applications of Snapping in AR

Annotation in AR

Industrial field: piping, wiring, interior design

Geocaching, mobile AR guide

Sketch based modeling in AR

Virtual object manipulation in AR

Xin (2008)

Zigsaw puzzle

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

5


Related work

Sketchpad (Sutherland 1963)

Beginning of direct manipulation GUI

Snapping while drawing vector graphics

Snap-dragging (Bier 1986)

Various alignment objects

Extend into 3D environment (1990)

Image Snapping (Gleicher 1995)

Using image features

Image editing applications

Sketchpad (Sutherland ’63)

Image Snapping (Gleicher ’95)

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

6


Snap-to-Feature Interface for AR

Real-time performance

Process sub-image in a window around interaction point

Better tracing result

Priority between different types of feature

Hysteresis

Robust to even more unstable view movement

Adaptive maximum snapping distance

Annotation friendliness

Implicit deactivation

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

7


Snap-to-Feature Interface for AR

Which feature to follow?

LDE = w·GDE (where w > 1)

LD C = GD C

D = ( 1 − f )·DI + f·DH

(where 0 ≤ f ≤ 1)

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

8


Snap-to-Feature Interface for AR

Robust to unstable view movement

Adaptively adjust the maximum snapping distance

proportionally to the view movement speed

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

9


Snap-to-Feature Interface for AR

Deactivating snapping temporarily

Explicit toggle switch

Implicit deactivation

when the stroke did not start with tracing (snapping)

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

10


Implementation

UMPC - Sony VAIO VGN-UX58LN

1.2 GHz Intel Core2Solo CPU / 1 GB Main Memory

Intel GMA 945 Integrated Graphics

4.5 inch touch screen

Video camera with 640x480 @ 30 fps

Software

OpenGL

ARToolkit (http://www.hitl.washington.edu/artoolkit/)

OpenCV (http://www.opencv.org)

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

11


Implementation

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

12


Results

Performance gain through sub-image processing

30

25

25fps

20

15

10

5

0

7fps

whole image

sub-image

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

13


Results

An example of tracing and annotation on a test figure

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

14


Results

Giving higher priority to corner points

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

15


Results

Considering spatial coherence

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

16


Results

Adaptive sub-image window size under shaky viewpoint

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

17


Results

Implicit deactivation of snapping for annotation

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

18


Results

All together

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

19


Results

Other examples

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

20


Results

Using snapping for modeling physical objects

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

21


Discussion &

Future work

Snap-to-Feature

Errors corrected appropriately, automatically

Helps users to perform tracing tasks with less error

Helps users to relax, causing less stress, making easier

Problems and Future work

Jittering and folding of trace lines

=> better tracking, curve fitting, distill out vertices

Support other features: centroids, principal axes ...

Apply to other interactions: 3D object manipulation

Formal user study

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

22


Conclusion

Snap-to-Feature Interface for Mobile AR

In real-time through sub-image processing

Higher priority to corner points

Spatial coherence

Robust to unstable view movements

Implicit deactivation

Helps users to interact in a more accurate, easy, and less

tired way

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

23


Thank you!

Gun A. Lee

endovert@etri.re.kr

ISMAR2010 AR Super Models Workshop, 13 Oct. 2010 / Gun A. Lee (endovert@etri.re.kr)

24

More magazines by this user
Similar magazines