Projective geometry for Computer Vision

Projective geometry for Computer Vision
Subhashis Banerjee
Department of Computer Science and Engineering
IIT Delhi
NIT, Rourkela
March 27, 2010
Subhashis Banerjee
Projective geometry for Computer Vision

Overview
◮ Pin-hole camera
◮ Why projective geometry?
◮ Reconstruction
Subhashis Banerjee
Projective geometry for Computer Vision

Computer vision geometry: main problems
Correspondence problem: Match image projections of a 3D
configuration.
Reconstruction problem: Recover the structure of the 3D
configuration from image projections.
Re-projection problem: Is a novel view of a 3D configuration
consistent with other views? (Novel view generation)
Subhashis Banerjee
Projective geometry for Computer Vision

An infinitely strange perspective
◮ Parallel lines in 3D space converge in images.
◮ The line of the horizon is formed by ‘infinitely’ distant points
(vanishing points).
◮ Any pair of parallel lines meet at a point on the horizon
corresponding to their common direction.
◮ All ‘intersections at infinity’ stay constant as the observer
moves.
Subhashis Banerjee
Projective geometry for Computer Vision

3D reconstruction from pin-hole projections
La Flagellazione di Cristo (1460) Galleria Nazionale delle Marche
by Piero della Francesca (1416-1492) (Robotics Research Group,
Oxford University, 2000)
Subhashis Banerjee
Projective geometry for Computer Vision

Pin-hole camera
◮ The effects can be modelled mathematically using the ‘linear
perspective’ or a ‘pin-hole camera’ (realized first by Leonardo?)
◮ If the world coordinates of a point are (X , Y , Z) and the
image coordinates are (x, y), then
◮ The model is non-linear.
x = fX /Z and y = fY /Z
Subhashis Banerjee
Projective geometry for Computer Vision

In terms of projective coordinates
where,
⎡
λ ⎣
⎡
⎣
x
y
1
x
y
1
⎤
⎡
⎦ = ⎣
⎤
⎦ ∈ P 2 and
are homogeneous coordinates.
f 0 0 0
0 f 0 0
0 0 1 0
⎡
⎢
⎣
X
Y
Z
1
⎡
⎤
⎦ ⎢
⎣
⎤
X
Y
Z
1
⎥
⎦ ∈ P3
⎤
⎥
⎦
Subhashis Banerjee
Projective geometry for Computer Vision

Euclidean and Affine geometries
◮ Given a coordinate system, n-dimensional real affine space is
the set of all points parameterized by x = (x 1 , . . . , x n ) t ∈ R n .
◮ An affine transformation is expressed as
x ′ = Ax + b
where A is a n × n (usually) non-singular matrix and b is a
n × 1 vector representing a translation.
◮ By SVD
A = UΣV T = (UV T )(VΣV T ) = R(θ)R(−φ)ΣR(φ)
where where
Σ =
[ ]
λ1 0
0 λ 2
Subhashis Banerjee
Projective geometry for Computer Vision

Euclidean and Affine geometries
◮ In the special case of when A is a rotation (i.e.,
AA t = A t A = I, then the transformation is Euclidean.
◮ An affine transformation preserves parallelism and ratios of
lengths along parallel directions.
◮ An Euclidean transformation, in addition to the above, also
preserves lengths and angles.
◮ Since an affine (or Euclidean) transformation preserves
parallelism it cannot be used to describe a pinhole projection.
Subhashis Banerjee
Projective geometry for Computer Vision

Spherical geometry
◮ The space S 2 :
S 2 = { x ∈ R 3 : ||x|| = 1 }
◮ lines in S 2 : Viewed as a set in R 3 this is the intersection of
S 2 with a plane through the origin. We will call this great
circle a line in S 2 . Let ξ be a unit vector. Then,
l = { x ∈ S 2 : ξ t x = 0 } is the line with pole ξ.
Subhashis Banerjee
Projective geometry for Computer Vision

Spherical geometry
◮ Lines in S 2 cannot be parallel. Any two lines intersect at a
pair of antipodal points.
◮ A point on a line:
◮ Two points define a line:
◮ Two lines define a point:
l· x = 0 or l T x = 0 or x T l = 0
l = p × q
x = l × m
Subhashis Banerjee
Projective geometry for Computer Vision

Projective geometry
◮ The projective plane P 2 is the set of all pairs {x, −x} of
antipodal points in S 2 .
◮ Two alternative definitions of P 2 , equivalent to the preceding
one are
1. The set of all lines through the origin in R 3 .
2. The set of all equivalence classes of ordered triples (x 1 , x 2 , x 3 )
of numbers (i.e., vectors in R 3 ) not all zero, where two vectors
are equivalent if they are proportional.
Subhashis Banerjee
Projective geometry for Computer Vision

Projective geometry
The space P 2 can be thought of as the infinite plane tangent to
the space S 2 and passing through the point (0, 0, 1) t .
Subhashis Banerjee
Projective geometry for Computer Vision

Projective geometry
◮ Let π : S 2 → P 2 be the mapping that sends x to {x, −x}.
The π is a two-to-one map of S 2 onto P 2 .
◮ A line of P 2 is a set of the form πl, where l is a line of S 2 .
Clearly, πx lies on πl if and only if ξ t x = 0.
◮ Homogeneous coordinates: In general, points of real
n-dimensional projective space, P n ,
are represented by n + 1 component column vectors
(x 1 , . . . , x n , x n+1 ) ∈ R n+1 such that at least one x i is non-zero
and (x 1 , . . . , x n , x n+1 ) and (λx 1 , . . . , λx n , λx n+1 ) represent the
same point of P n for all λ ≠ 0.
◮ (x 1 , . . . , x n , x n+1 ) is the homogeneous representation of a
projective point.
Subhashis Banerjee
Projective geometry for Computer Vision

Canonical injection of R n into P n
◮ Affine space R n can be embedded in P n by
(x 1 , . . . , x n ) → (x 1 , . . . , x n , 1)
◮ Affine points can be recovered from projective points with
x n+1 ≠ 0 by
(x 1 , . . . , x n ) ∼ ( x 1
x n+1
, . . . ,
x n
x n+1
, 1) → ( x 1
x n+1
, . . . ,
x n
x n+1
)
◮ A projective point with x n+1 = 0 corresponds to a point at
infinity.
◮ The ray (x 1 , . . . , x n , 0) can be viewed as an additional ideal
point as (x 1 , . . . , x n ) recedes to infinity in a certain direction.
For example, in P 2 ,
lim (X /T , Y /T , 1) = lim (X , Y , T ) = (X , Y , 0)
T →0 T →0
Subhashis Banerjee
Projective geometry for Computer Vision

Lines in P 2
◮ A line equation in R 2 is
a 1 x 1 + a 2 x 2 + a 3 = 0
◮ Substituting by homogeneous coordinates x i = X i /X 3 we get
a homogeneous linear equation
(a 1 , a 2 , a 3 )· (X 1 , X 2 , X 3 ) =
3∑
a i X i = 0, X ∈ P 2
i=1
◮ A line in P 2 is represented by a homogeneous 3-vector
(a 1 , a 2 , a 3 ).
◮ A point on a line: a· X = 0 or a T X = 0 or X T a = 0
◮ Two points define a line: l = p × q
◮ Two lines define a point: x = l × m
Subhashis Banerjee
Projective geometry for Computer Vision

The line at infinity
◮ The line at infinity (l ∞ ): is the line of equation X 3 = 0.
Thus, the homogeneous representation of l ∞ is (0, 0, 1).
◮ The line (u 1 , u 2 , u 3 ) intersects l ∞ at the point (−u 2 , u 1 , 0).
◮ Points on l ∞ are directions of affine lines in the embedded
affine space (can be extended to higher dimensions).
Subhashis Banerjee
Projective geometry for Computer Vision

Conics in P 2
A conic in affine space (inhomogeneous coordinates) is
ax 2 + bxy + cy 2 + dx + ey + f = 0
Homogenizing this by replacements x = X 1 /X 3 and y = Y 1 /Y 3 , we
obtain
aX 2 1 + bX 1 X 2 + cX 2 2 + dX 1 X 3 + eX 2 X 3 + fX 2 3 = 0
which can be written in matrix notation as X T CX = 0 where C is
symmetric and is the homogeneous representation of a conic.
Subhashis Banerjee
Projective geometry for Computer Vision

Conics in P 2
◮ The line l tangent to a conic C at any point x is given by
l = Cx.
◮ x t Cx = 0 =⇒ (C −1 l) t C((C −1 l) = l t C −1 l = 0
(because C −t = C −1 ). This is the equation of the dual conic.
Subhashis Banerjee
Projective geometry for Computer Vision

Conics in P 2
◮ The degenerate conic of rank 2 is defined by two line l and m
as
C = lm t + ml t
Points on line l satisfy l t x = 0 and are hence on the conic
because (x t l)(m t x) + (x t m)(l t x) = 0. (Similarly for m).
The dual conic xy t + yx t represents lines passing through x
and y.
Subhashis Banerjee
Projective geometry for Computer Vision

Projective basis
Projective basis: A projective basis for P n is any set of n + 2
points no n + 1 of which are linearly dependent.
Canonical basis:
⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤
1 0
0 1
0
⎢ ⎥
⎣ . ⎦ , 1
⎢ ⎥
⎣ . ⎦ , . . .
0
⎢ ⎥
⎣ . ⎦ ,
1
⎢ ⎥
⎣ . ⎦
0 0
1 1
} {{ } } {{ } } {{ }
points at infinity along each axis origin unit point
Subhashis Banerjee
Projective geometry for Computer Vision

Projective basis
Change of basis: Let e 1 , e 2 , . . . , e n+1 , e n+2 be the standard basis
and a 1 , a 2 , . . . , a n+1 , a n+2 be any other basis. There
exists a non-singular transformation [T] (n+1)×(n+1)
such that:
Te i = λ i a i , ∀i = 1, 2. . . . , n + 2
T is unique up to a scale.
Subhashis Banerjee
Projective geometry for Computer Vision

Homography
The invertible transformation T : P n → P n is called a projective
transformation or collineation or homography or perspectivity
and is completely determined by n + 2 point correspondences.
◮ Preserves straight lines and cross ratios
◮ Given four collinear points A 1 , A 2 ,A 3 and A 4 , their cross
ratio is defined as
A 1 A 3 A 2 A 4
A 1 A 4 A 2 A 3
◮ If A 4 is a point at infinity then the cross ratio is given as
A 1 A 3
A 2 A 3
◮ The cross ratio is independent of the choice of the projective
coordinate system.
Subhashis Banerjee
Projective geometry for Computer Vision

Homography
Subhashis Banerjee
Projective geometry for Computer Vision

Homography
Subhashis Banerjee
Projective geometry for Computer Vision

Projective mappings of lines
◮ If the points x i lie on the line l, we have l T x i = 0.
◮ Since, l T H −1 Hx i = 0 the points Hx i all lie on the line H −T l.
◮ Hence, if points are transformed as x ′ i = Hx i , lines are
transformed as l ′ = H −T l.
Subhashis Banerjee
Projective geometry for Computer Vision

Projective mappings of conics
◮ Note that a conic is represented (homogeneously) as
x T Cx = 0
◮ Under a point transformation x ′ = Hx the conic becomes
x T Cx = x ′ T [H −1 ] T CH −1 x ′ = x ′ T H −T CH −1 x ′ = 0
◮ This is the quadratic form of x ′T C ′ x ′ with C ′ = H −T CH −1 .
This gives the transformation rule for a conic.
Subhashis Banerjee
Projective geometry for Computer Vision

The affine subgroup
In an affine space A n an affine transformation defines a
correspondence X ↔ X ′ given by:
X ′ = AX + b
where X, X ′ and b are n-vectors, and A is an n × n matrix.
Clearly this is a subgroup of the projective group. Its projective
representation is
[ ] C c
T =
0 T n t 33
where A = 1
t 33
C and b = 1
t 33
c.
Subhashis Banerjee
Projective geometry for Computer Vision

Affine transformations preserve the plane/line at infinity
[ A b
0 t 1
] ⎛ ⎝
x 1
x 2
0
⎞ ⎛ ( ) ⎞
⎠ = ⎝ A x1
x 2
⎠
0
A general projective transformation moves points at infinity to
finite points.
Subhashis Banerjee
Projective geometry for Computer Vision

The Euclidean subgroup
◮ The absolute conic: The conic Ω ∞ is intersection of the
quadric of equation:
∑
n+1
xi
2
i=1
= x n+1 = 0 with π ∞
◮ In a metric frame π ∞ = (0, 0, 0, 1) T , and points on Ω ∞ satisfy
X1 2 + X 2 2 + X 3
3 }
= 0
X 4
◮ For directions on π ∞ (with X 4 = 0), the absolute conic Ω ∞
can be expressed as
(X 1 , X 2 , X 3 )I(X 1 , X 2 , X 3 ) T = 0
◮ The absolute conic, Ω ∞ , is fixed under a projective
transformation H if and only if H is an Euclidean
transformation.
Subhashis Banerjee
Projective geometry for Computer Vision

Affine calibration of a plane
Subhashis Banerjee
Projective geometry for Computer Vision

Affine calibration of a plane
If the imaged line at infinity is l = (l 1 , l 2 , l 3 ) t , then provided l 3 ≠ 0
a suitable projective transformation that maps l back to l ∞ is
⎡ ⎤
1 0 0
H = ⎣ 0 1 0 ⎦ H A
l 1 l 2 l 3
Subhashis Banerjee
Projective geometry for Computer Vision

Reconstruction
Camera recovery
Metrology
Subhashis Banerjee
Projective geometry for Computer Vision

Surfaces of revolution
Subhashis Banerjee
Projective geometry for Computer Vision

Modeling of structured scenes
Subhashis Banerjee
Projective geometry for Computer Vision

A walkthrough
Subhashis Banerjee
Projective geometry for Computer Vision