Projective Geometry - Institute for Computer Graphics and Vision

Institut für Elektrische Meßtechnik und Meßsignalverarbeitung 

Augmented Reality 

• Camera(s) 

• Real scene, scene coordinates 

C 

y C 

x C 

R 1 , t 1 

z C 

• Real table 

• Augmented plant 

R 2 , t 2 

Z 

x y V 

V 

z V 

• Visualization (screen, HMD) 

X 

Y 

Professor Augmented Horst Reality Cerjak, VU 19.12.20051 Projective Geometry Axel Pinz 

7.3.2011 

1


Example 1: ARToolkit 


7.3.2011 

2


Example 2: Structure + Motion [Schweighofer] 


7.3.2011 

3


Pinhole Camera 

“principal” 

point (u 0 ,v 0 ) 

v 

u 

p(u,v) 

image plane π i (u,v): z = -f 

f 

y 

C 

x 

z 

P(x,y,z) 

“optical axis” 

P’(x’,y’,z’) 

“focal length” f 

• “real” camera 

• “Pinhole” C … “center of projection” 

• 2D projection 3D scene 

• p(u,v) ↔ line of sight = viewing direction 


7.3.2011 

4


Projective Geometry 

P’’ 

!!! 

y x 

z 

C 

π 0 ... z = 0 

p(u,v) 

y x 

(u 0 ,v 0 ) 

v 

u 

π i ... z = f 

P(x,y,z) 

P’ 

“projective” camera, “normalized” camera: f = 1 

1 stationary camera 1 coordinate system (x,y,z) 

camera-centered coordinate system ≡ scene coordinate system 

Only points in π 0 are not projected to π i 


7.3.2011 

5


Projective Images: Examples, Properties 

• Impression of depth in images 

• Parallel lines meet at infinity 

• “infinity” is projected to finite 

location in the image 

• “horizon” 

• “points at infinity”, … 

[Triggs and Mohr] 


7.3.2011 

6


Projective Images: Scaling 


7.3.2011 

7


Projective Images: Foreshortening 


7.3.2011 

8


Projective Images: Parallel Lines Meet 

[Sonka, Hlavac, Boyle] 


7.3.2011 

9


Projective Geometry 

• points in π 1 

• straight lines of sight 

• projective reconstruction 

• geometry, precise 

• known correspondences 

vs. Computer Vision 

• discrete pixels in π i 

• sampling theorem 

• lens distortion, aperture, 

depth of field 

• “oriented” projective rec. 

“in front of camera” 

• inherently imprecise 

estimation, minimization 

• “outliers” robustness 


7.3.2011 

10


Example: Stereo Reconstruction 

~ 

P 

P 

C 1 

C 2 

• projective geometry 

• computer vision 


7.3.2011 

11


Algebraic Projective Geometry (1) 

• A unified geometric + algebraic framework 

• Point 

x0 

 

p 

x0, 

y0 

y 

0 

T 

• Line 

y 

kx 

d 

0 

ax by 

 

c 

 

a 

 

 

b 

 

c 

x 

y 

1 

 

 

 

 

 

 

0 

 

l 

 

x 

 

0 

 

l 

 

 

 

 

 

 

a 

b 

c 

 

 

 

 

 

 

x 

 

 

 

 

 

 

x 

 

y 

1 

 


7.3.2011 

12


Algebraic Projective Geometry (2) 

• Duality 

point ↔ line 

p 2 

l 1 

2 

 

p 

 

l 1 

l 2 

 

l 

 

p 1 

p 

l 2 

 

p 1 

• Unified approach: projective n-space P n 

point 

 

x x 

, , 

x T 

1 n 1 

(n+1) - vector 


7.3.2011 

13


Homogeneous Coordinates in P n 

a 

 

l b 

~ 

 

c 

a 

 

kb 

 

c 

ax by c 0 

( ka) 

x ( kb) 

y kc 

0 k 

0 

Equivalence class of vectors 

3 

R 

0 

 

0 

 

0 

Homogeneous coordinates 

x 

inhomogeneous 

y 

forms P 2 … “projective plane” 


 

 

 

 

 

x 

x 

x 

1 

2 

3 

 

 

, but only 2 DoF 

 

 

7.3.2011 

14


Equivalence Class of Vectors 

Without further knowledge, such situations 

cannot be distingushed ! 

A further example: Equivalence of a toy car, closeup shot, and 

real car, distant shot 


7.3.2011 

15


• Point 

 

x 

 

The Projective Plane (1) 

 

l 1 

l 2 

• Line 

 

l 

 

 

x 1 

x 2 

• “ideal” points 

 

 

 

 

 

• “line at infinity” 


x 

x 

1 

2 

0 

 

 

 

 

 

l 

 

intersection of 

parallel lines ! 

treated like any point x 3 ≠0 

0 

 

0 

 

1 

„Fernpunkte“ 

the plane’s “horizon” 

7.3.2011 

16



• Adding the ideal 

points to R 2 leads 

to the projective 

plane P 2 

[Hartley+Zisserman] 

• Covers all homogeneous 

x1 

 

 

coordinates 

 

 

 

x 

x 

2 

3 

 

 

 

0 

 

0 

 

0 


7.3.2011 

17



vanishing point, 

„Fluchtpunkt“ = 

Bild eines 

„Fernpunktes“ 

π 

Image of the “horizon” of π, 

“line at infinity” of π 

• Projective geometry can map infinitely far points / lines to finite ones 

• No difference between finite and infinite 

• e.g. hyperbola is one continuous conic 


7.3.2011 

18

• Points 

• Planes 

• 


There is also Projective Space P 3 … 

x1 

 

 

x2 

 

p x1 

x2 

x3 

x4 

x 

3 

 

x 

4 

 

a 

p a 

a 

 

b 

 

c 

 

d 

 

 

 

a 

b 

c 

d 

T 

0 point plane 

T 

• Lines: 4 DoF 

 

2 points p 

T 

p 

1 

L 

T 

p 

2 

 

l : p 

p 

1 

• Dual line L*: 

1 

2 

, 

 

p 

LL 

duality point ↔ plane duality L ↔ L* 

2 

* T 

02 

2 


7.3.2011 

19


Projective Transformations in P n 

• “projective transformation” = “collineation” = 

“projectivity” = “homography” H 

• Invertible mapping P n → P 

 

n 

 

• x1, x2, 

x3 

lie on a line Hx1 

, Hx2 

, Hx3 

lie on a line 

„geradentreue“ Abbildung 

• (n+1) x (n+1) matrix 

• In P 2 : x1 

' h11 

h12 

h13 

x1 

 

 

x' 

 

 

 

 

 

x 

x 

2 

3 

 

' Hx 

 

' 

 

 

 

x 

 

 

x 

• H has (n+1) 2 -1 DoF, H is non-singular 

 

 

 

 

h 

h 

21 

31 

h 

h 

22 

32 

h 

h 

23 

33 

2 

3 

 

 

 

 


7.3.2011 

20


Projective Transformations in P 2 

• Translation 

• Rotation 

• Scaling 

• Any combination, e.g. 

1 

0 tx 

 

 

T 0 

1 tx 

x' 

Tx 

 

0 

0 1 

cos 

sin 

0 

 

 

R sin 

cos 

0 

x' 

Rx 

 

 

0 0 1 

sx 

0 0 

 

S 0 sy 

0 

x' 

Sx 

 

0 0 1 

 

M SRT x' 

SRTx 

 

 

Mx 


7.3.2011 

21


A Remark on Conics 

• 2 nd degree equation in the plane 

ax 

• Homog. coord: 

ax 

• Conic C: 

2 

2 

1 

bxy 

bx 

 

x on C 

1 

x 

cy 

2 

 

• Five DoF, 5 points define a conic 

2 

cx 

 

x 

T 

dx ey 

2 

2 

 

Cx 

x 

 

 

 

dx 

0 , 

x 

1 

1 

x 

3 

/ 

 

x 

3 

f 

ex 

2 

 

x 

y 

3 

0 

 

 

a 

 

C b / 2 

 

d 

/ 2 

x 

fx 

2 

2 

3 

/ 

c 

 

x 

b / 2 

e / 2 

3 

0 

d 

/ 

e / 

f 

2 

 

2 

 

 


7.3.2011 

22


Back to Homographies – Examples (1) 

Mapping between planes 


central projection may be expressed by x’=Hx 


7.3.2011 

23



Removing projective distortion 



7.3.2011 

24





7.3.2011 

25


Transformation for Points, Lines, Conics 

• Point 

 

x' Hx 

• Line 

l 

' 

1 

H 

 

T l 

, 

H 

T 

 

 

1 

 

T 

 

 

H H 

T 1 

• Conic 

C' 

 

H 

T 

CH 

 


7.3.2011 

26


A Hierarchy of Transformations / Geometries (1) 

• Isometric / Euclidean 

H 

E 

– Invariants: length, angle, area 

• Similarity 

H 

S 

R 

 

 

T 

0 

 

t 

, 

1 

 

3 DoF: , 

t 

 

sR 

t 

 

, 4 DoF: 

T 

0 1 

 

– Invariants: ratios of length / areas, angle, parallel lines 

x 

t 

y 

s, , 

t 

x 

t 

y 


7.3.2011 

27



• Affine: 

– A 

H 

A 

a 

 

a 

 

0 

11 

21 

a 

a 

12 

22 

0 

R( 

) 

R( 

) 

DR( 

), 

tx 

 

A 

t 

 

y T 

0 

1 

 

 

 

D 

 

1 

0 

 

t 

 

1 

 

0 

 

 

2 

– 6 DoF: 2 x scale λ 1 ,λ 2 ; 2 x rot. θ,Ф; 2 x translation 

– Invariants: parallel lines, ratios of parallel lengths, 

ratios of areas 


7.3.2011 

28



• Projective: 

H 

h 

 

h 

 

h 

11 

P 21 22 23 T 

31 

h 

h 

h 

12 

32 

h 

h 

h 

13 

33 

 

A 

 

v 

 

 

t 

 

v 

 

– 8 DoF: 2 x scale λ 1 ,λ 2 ; 2 x rot. θ,Ф; 2 x translation; 

2 x line at infinity 

– Invariant: Cross-ratio CR of 4 collinear points 

CR 

 

AB. 

CD 

AD. 

BC 

A B C D 


7.3.2011 

29



In 2D, a square 

transforms to: 

Projective 

8dof 

Affine 

6dof 

Similarity 

4dof 

Euclidean 

3dof 

h 

 

 

h 

 

h 

11 

21 

31 

a 

 

 

a 

 

0 

11 

21 

sr 

 

 

sr 

 

0 

r 

 

 

r 

 

0 

11 

21 

11 

21 

h 

h 

h 

a 

a 

r 

r 

12 

22 

32 

12 

22 

0 

sr 

sr 

12 

22 

0 

12 

0 

22 

h 

h 

h 

13 

23 

33 

tx 

 

t 

 

y 

 

1 

tx 

 

t 

 

y 

 

1 

 

 

 

 

tx 

 

t 

 

y 

 

1 


7.3.2011 

30



Projective 

15dof 

A 

T 

v 

t 

v 

 

In 3D, a cube 

transforms to: 

Affine 

12dof 

A 

T 

0 

t 

1 

 

Similarity 

7dof 

s 

R 

T 

0 

t 

1 

 

Euclidean 

6dof 

R 

T 

0 

t 

1 

 

 


7.3.2011 

31


Stratification 

In AR, we take perspective images, 

but we require metric (Euclidean) reconstruction! 

How? 

The stratification of 3D geometry [Pollefeys 2.2] 


7.3.2011 

32


Stratification of 2D / 3D Geometry 

Many possibilities, many approaches 

Examples: 

• Known scenes: 

– Known . directions 

– Known points, lines, planes at ∞ 

– Known lengths in the scene 

• Unknown scenes: 

– Image of the Absolute Conic “IAC” (“self-calibration”) 

– Known camera intrinsics 

Camera calibration + relative orientation 

Multiview geometry, structure+motion 


7.3.2011 

33


Stratification Examples (1) 

• Known points, line at infinity 

v 1 v 2 

l ∞ 

l 1 

l 3 

l 2 l 4 

v 

v 

l 

l 

 

2 3 

l4 

 

1 1 

l2 

l 

 

 

v 1 

v 2 

perspective 


affine 

7.3.2011 

34



• Known · 

directions 

affine 

metric 

(similarity, unknown scale) 


7.3.2011 

35



• Known plane at infinity 

perspective 

affine 


7.3.2011 

36



• Known · 

directions 

affine 

metric 

(similarity, unknown scale) 


7.3.2011 

37



• Known lengths 

[Pollefeys IJCV’99] 

metric 

(similarity, 

unknown scale) 

metric 

(Euclidean, 

known scale) 


7.3.2011 

38


• ARToolkit 


perspective 

metric 

(Euclidean, 

known scale) 


7.3.2011 

39


“Video AR” is (rather) simple 

• Known artificial targets / markers 

• Uncalibrated perspective camera 

– But: collineation required 

– Problems when e.g. strong lens distortions 

• Augmentation of the video frames 

• Examples 

– Artoolkit 

– Kutulakos 

• Can be related to scene coordinates, but 

requires “ground truth” for markers 


7.3.2011 

40


ARToolkit Demo ISAR 2000 

observer’s view 

immersive view 


7.3.2011 

41


Kutulakos’ “Calibration-Free AR” 

[IEEE Trans. Visualization and Graphics 1998] 


7.3.2011 

42


Field Maintenance Support [ARVIKA] 


7.3.2011 

43


Scene Structure + Camera Motion 

(the harder, but more general approach to AR) 

• Many possible approaches 

• Monocular, calibrated, known “natural” 

landmarks [Ribo] 

• Stereo, calibrated [Schweighofer] 

• Monocular, calibrated [Klein+Murray] 

• Monocular, uncalibrated [Pollefeys] 

– not (yet?) in real time ! 

Kinect ! calibration ! 

unknown 

scene, 

unknown 

“natural” 

landmarks 


7.3.2011 

44

Projective Geometry - Institute for Computer Graphics and Vision

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?