Lecture 12 - Stanford Vision Lab

Lecture 12:Local Descriptors, SIFT& Single Object RecognitionProfessor Fei‐Fei LiStanford Vision LabFei-Fei LiLecture 12 -15‐Nov‐11

What we will learn today• Local descriptors– SIFT (Problem Set 3 (Q2))– An assortment of other descriptors (Problem Set 3(Q4))• Recognition and matching with local features(Problem Set 3 (Q3))– Matching objects with local features: David Lowe– Panorama stitching: Brown & Lowe– ApplicationsFei-Fei LiLecture 12 -25‐Nov‐11

Local Descriptors• We know how to detect points• Next question:Last lecture (#11)How to describe them for matching?This lecture (#12)?Point descriptor should be:1. Invariant2. DistinctiveSlide credit: Kristen GraumanFei-Fei LiLecture 12 -35‐Nov‐11

Orientation Normalization: Computation[Lowe, SIFT, 1999]• Compute orientation histogram• Select dominant orientation• Normalize: rotate to fixed orientation0 2Slide adapted from David LoweFei-Fei LiLecture 12 -55‐Nov‐11

The Need for Invariance• Up to now, we had invariance to– Translation– Scale– Rotation• Not sufficient to match regions under viewpoint changes– For this, we need also affine adaptationSlide credit: Tinne TuytelaarsFei-Fei LiLecture 12 -65‐Nov‐11

• Problem:Affine Adaptation– Determine the characteristic shape of the region.– Assumption: shape can be described by “local affine frame”.• Solution: iterative approach– Use a circular window to compute second moment matrix.– Compute eigenvectors to adapt the circle to an ellipse.– Recompute second moment matrix using new window and iterate…Slide adapted from Svetlana LazebnikFei-Fei LiLecture 12 -75‐Nov‐11

Iterative Affine Adaptation1. Detect keypoints, e.g. multi‐scale Harris2. Automatically select the scales3. Adapt affine shape based on second order moment matrix4. Refine point locationK. Mikolajczyk and C. Schmid, Scale and affine invariant interest point detectors, IJCV 60(1):63‐86, 2004.Slide credit: Tinne TuytelaarsFei-Fei LiLecture 12 -85‐Nov‐11

Affine Normalization/Deskewingrotaterescale• Steps– Rotate the ellipse’s main axis to horizontal– Scale the x axis, such that it forms a circleSlide credit: Tinne TuytelaarsFei-Fei LiLecture 12 -95‐Nov‐11

Affine Adaptation ExampleScale‐invariant regions (blobs)Slide credit: Svetlana LazebnikFei-Fei LiLecture 12 -105‐Nov‐11

Affine Adaptation ExampleAffine‐adapted blobsSlide credit: Svetlana LazebnikFei-Fei LiLecture 12 -115‐Nov‐11

Summary: Affine‐Inv. Feature ExtractionExtract affine regionsNormalize regionsEliminate rotationalambiguityComparedescriptorsSlide credit: Svetlana LazebnikFei-Fei LiLecture 12 -125‐Nov‐11

Feature Descriptors• Disadvantage of patches as descriptors:– Small shifts can affect matching score a lot• Solution: histograms0 2 Slide credit: Svetlana LazebnikFei-Fei LiLecture 12 -165‐Nov‐11

Feature Descriptors: SIFT• Scale Invariant Feature Transform• Descriptor computation:– Divide patch into 4x4 sub‐patches: 16 cells– Compute histogram of gradient orientations (8 reference angles) for allpixels inside each sub‐patch– Resulting descriptor: 4x4x8 = 128 dimensionsDavid G. Lowe. "Distinctive image features from scale‐invariant keypoints.” IJCV 60 (2), pp. 91‐110, 2004.Slide credit: Svetlana LazebnikFei-Fei LiLecture 12 -175‐Nov‐11

Overview: SIFT• Extraordinarily robust matching technique– Can handle changes in viewpoint up to ~60 deg. out‐of‐plane rotation– Can handle significant changes in illumination• Sometimes even day vs. night (below)– Fast and efficient—can run in real time– Lots of code available• http://people.csail.mit.edu/albert/ladypack/wiki/index.php/Known_implementations_of_SIFTSlide credit: Steve SeitzFei-Fei LiLecture 12 -185‐Nov‐11

Working with SIFT Descriptors• One image yields:– n 128‐dimensional descriptors: eachone is a histogram of the gradientorientations within a patch• [n x 128 matrix]– n scale parameters specifying thesize of each patch• [n x 1 vector]– n orientation parameters specifyingthe angle of the patch• [n x 1 vector]– n 2D points giving positions of thepatches• [n x 2 matrix]Slide credit: Steve SeitzFei-Fei LiLecture 12 -195‐Nov‐11

Other local descriptors:Gray‐scale intensityNormalize11x11 patchProjection ontoPCA basisc 1c 2………..c 15Fei-Fei LiLecture 12 -215‐Nov‐11

Other local descriptors:Steerable filtersFei-Fei LiLecture 12 -225‐Nov‐11

Other local descriptors:Gray‐scale intensityFei-Fei LiLecture 12 -235‐Nov‐11

Other local descriptors:GLOHFei-Fei LiLecture 12 -245‐Nov‐11

Other local descriptors:Shape context[Belongie et al. 2002]Fei-Fei LiLecture 12 -255‐Nov‐11

Other local descriptors:Geometric Blur[Berg et al. 2001]Fei-Fei LiLecture 12 -265‐Nov‐11

Other local descriptors:Geometric BlurFei-Fei LiLecture 12 -275‐Nov‐11

http://www.robots.ox.ac.uk/~vgg/research/affine/detectors.html#binariesFei-Fei LiLecture 12 -285‐Nov‐11

• AdvantagesValue of Local Features– Critical to find distinctive and repeatable localregions for multi‐view matching.– Complexity reduction via selection of distinctivepoints.– Describe images, objects, parts without requiringsegmentation; robustness to clutter & occlusion.– Robustness: similar descriptors in spite ofmoderate view changes, noise, blur, etc.Fei-Fei LiLecture 12 -29 5‐Nov‐11

What we will learn today• Local descriptors– SIFT– An assortment of other descriptors• Recognition and matching with local features(Problem Set 3 (Q3))– Matching objects with local features: David Lowe– Panorama stitching: Brown & Lowe– ApplicationsFei-Fei LiLecture 12 -305‐Nov‐11

Recognition with Local Features• Image content is transformed into local features that areinvariant to translation, rotation, and scale• Goal: Verify if they belong to a consistent configurationLocal Features,e.g. SIFTSlide credit: David LoweFei-Fei LiLecture 12 -315‐Nov‐11

Finding Consistent Configurations• Global spatial models– Generalized Hough Transform [Lowe99]– RANSAC [Obdrzalek02, Chum05, Nister06]– Basic assumption: object is planar• Assumption is often justified in practice– Valid for many structures onbuildings– Sufficient for small viewpointvariations on 3D objectsFei-Fei LiLecture 12 -325‐Nov‐11

Reminder: Hough Transform• Origin: Detection of straight lines in clutter– Basic idea: each candidate point votesfor all lines that it is consistent with.– Votes are accumulated in quantized array– Local maxima correspond to candidate lines• Representation of a line– Usual form y = a x + b has a singularity around 90º.– Better parameterization: x cos() + y sin() = yxyρxθFei-Fei LiLecture 12 -335‐Nov‐11

3D Object Recognition• Gen. Hough T. for Recognition[Lowe99]– Typically only 3 feature matchesneeded for recognition– Extra matches providerobustness– Affine model can be used forplanar objectsFei-Fei LiLecture 12 -345‐Nov‐11

View Interpolation• Training– Training views from similarviewpoints are clusteredbased on feature matches.– Matching features betweenadjacent views are linked.• Recognition– Feature matches may bespread over severaltraining viewpoints. Use the known links to “transfer votes” to otherviewpoints.Fei-Fei LiLecture 12 -35 5‐Nov‐11

Recognition Using View InterpolationFei-Fei LiLecture 12 -36 5‐Nov‐11

Location RecognitionTrainingFei-Fei LiLecture 12 -375‐Nov‐11

What we will learn today• Local descriptors– SIFT– An assortment of other descriptors• Recognition and matching with local features(Problem Set 3 (Q3))– Matching objects with local features: David Lowe– Panorama stitching: Brown & Lowe– ApplicationsFei-Fei LiLecture 12 -385‐Nov‐11

Introduction• Are you getting the whole picture?– Compact Camera FOV = 50 x 35°Slide credit: Matthew BrownFei-Fei LiLecture 12 -395‐Nov‐11

Introduction• Are you getting the whole picture?– Compact Camera FOV = 50 x 35°– Human FOV = 200 x 135°Slide credit: Matthew BrownFei-Fei LiLecture 12 -405‐Nov‐11

Introduction• Are you getting the whole picture?– Compact Camera FOV = 50 x 35°– Human FOV = 200 x 135°– Panoramic Mosaic = 360 x 180°Slide credit: Matthew BrownFei-Fei LiLecture 12 -415‐Nov‐11

Why “Recognising Panoramas”?• 1D Rotations ()– Ordering matching imagesSlide credit: Matthew BrownFei-Fei LiLecture 12 -425‐Nov‐11

Why “Recognising Panoramas”?• 1D Rotations ()– Ordering matching imagesSlide credit: Matthew BrownFei-Fei LiLecture 12 -435‐Nov‐11

Why “Recognising Panoramas”?• 1D Rotations ()– Ordering matching imagesSlide credit: Matthew BrownFei-Fei LiLecture 12 -445‐Nov‐11

Why “Recognising Panoramas”?• 1D Rotations ()– Ordering matching images• 2D Rotations (, )– Ordering matching imagesSlide credit: Matthew BrownFei-Fei LiLecture 12 -455‐Nov‐11

Why “Recognising Panoramas”?• 1D Rotations ()– Ordering matching images• 2D Rotations (, )– Ordering matching imagesFei-Fei LiLecture 12 -465‐Nov‐11

Why “Recognising Panoramas”?• 1D Rotations ()– Ordering matching images• 2D Rotations (, )– Ordering matching imagesFei-Fei LiLecture 12 -475‐Nov‐11

Why “Recognising Panoramas”?Slide credit: Matthew BrownFei-Fei LiLecture 12 -485‐Nov‐11

Algorithm (Brown & Lowe, 2003)1. Feature Matchinga) SIFT Featuresb) Nearest Neighbor Matching2. Image Matching3. Bundle Adjustment4. Multi‐band BlendingFei-Fei LiLecture 12 -495‐Nov‐11

Nearest Neighbor Matching of SIFT Features0 2 • Find k‐NN for each feature– k number of overlapping images (we use k = 4)• Use k‐d tree– k‐d tree recursively bi‐partitions data at mean in thedimension of maximum variance– Approximate nearest neighbors found in O(nlogn)Slide credit: Matthew BrownFei-Fei LiLecture 12 -505‐Nov‐11

Algorithm (Brown & Lowe, 2003)1. Feature Matching2. Image Matchinga) RANSAC for Homographyb) Probabilistic model for verification3. Bundle Adjustment4. Multi‐band BlendingFei-Fei LiLecture 12 -515‐Nov‐11

Slide credit: Matthew BrownFei-Fei LiRANSAC for HomographyLecture 12 -5‐Nov‐1152

Slide credit: Matthew BrownFei-Fei LiRANSAC for HomographyLecture 12 -5‐Nov‐1153

Slide credit: Matthew BrownFei-Fei LiRANSAC for HomographyLecture 12 -545‐Nov‐11

Probabilistic model for verificationSlide credit: Matthew BrownFei-Fei LiLecture 12 -555‐Nov‐11

Probabilistic model for verification• Compare probability that this set of RANSACinliers/outliers was generated by a correct/false imagematch– n i = #inliers, n f = #features– p 1 = p(inlier | match), p 0 = p(inlier | ~match)– p min = acceptance probability• Choosing values for p 1 , p 0 and p minSlide credit: Matthew BrownFei-Fei LiLecture 12 -565‐Nov‐11

Slide credit: Matthew BrownFei-Fei LiFinding the panoramasLecture 12 -575‐Nov‐11

Slide credit: Matthew BrownFei-Fei LiFinding the panoramasLecture 12 -585‐Nov‐11

Slide credit: Matthew BrownFei-Fei LiFinding the panoramasLecture 12 -595‐Nov‐11

Slide credit: Matthew BrownFei-Fei LiFinding the panoramasLecture 12 -5‐Nov‐1160

Algorithm (Brown & Lowe, 2003)1. Feature Matching2. Image Matching3. Bundle Adjustment (extra reading: H & Z, Chp 18)a) Error function4. Multi‐band BlendingFei-Fei LiLecture 12 -615‐Nov‐11

Error function• Sum of squared projection errors– n = #images– I(i) = set of image matches to image i– F(i, j) = set of feature matches between images i,j– r ijk = residual of k th feature match between images i,j• Robust error functionSlide credit: Matthew BrownFei-Fei LiLecture 12 -625‐Nov‐11

Bundle Adjustment• New images initialised with rotation, focallength of best matching imageSlide credit: Matthew BrownFei-Fei LiLecture 12 -635‐Nov‐11

Bundle Adjustment• New images initialised with rotation, focallength of best matching imageSlide credit: Matthew BrownFei-Fei LiLecture 12 -645‐Nov‐11

Algorithm (Brown & Lowe, 2003)1. Feature Matching2. Image Matching3. Bundle Adjustment4. Multi‐band BlendingFei-Fei LiLecture 12 -655‐Nov‐11

Multi‐band Blending• Burt & Adelson 1983– Blend frequency bands over range Slide credit: Matthew BrownFei-Fei LiLecture 12 -665‐Nov‐11

2‐band BlendingLow frequency ( > 2 pixels)High frequency ( < 2 pixels)Slide credit: Matthew BrownFei-Fei LiLecture 12 -675‐Nov‐11

Linear BlendingFei-Fei LiLecture 12 -685‐Nov‐11

2‐band BlendingFei-Fei LiLecture 12 -695‐Nov‐11

Slide credit: Matthew BrownFei-Fei LiResultsLecture 12 -5‐Nov‐1170

ResultsiPhone versionavailablehttp://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html[Brown, Szeliski, and Winder, 2005]Fei-Fei LiLecture 12 -715‐Nov‐11

Applications of Local Invariant Features• Wide baseline stereo• Motion tracking• Mobile robot navigation• 3D reconstruction• Recognition– Specific objects– Textures– Categories• …Fei-Fei LiLecture 12 -725‐Nov‐11

Wide‐Baseline StereoImage from T. Tuytelaars ECCV 2006 tutorialFei-Fei LiLecture 12 -735‐Nov‐11

Recognition of Specific Objects, ScenesSchmid and Mohr 1997Sivic and Zisserman, 2003Rothganger et al. 2003 Lowe 2002Slide credit: Kristen GraumanFei-Fei LiLecture 12 -745‐Nov‐11

Recognition of CategoriesConstellation modelBags of wordsWeber et al. (2000)Fergus et al. (2003)Fei‐Fei et al. (2003)Csurka et al. (2004)Fei‐Fei & Perona (2005)Dorko & Schmid (2005)Sivic et al. (2005)Lazebnik et al. (2006), …Slide credit: Svetlana LazebnikFei-Fei LiLecture 12 -755‐Nov‐11

References and Further Reading• More details on homography estimation can be foundin Chapter 4.7 of– R. Hartley, A. ZissermanMultiple View Geometry in Computer Vision2nd Ed., Cambridge Univ. Press, 2004• Details about the DoG detector andthe SIFT descriptor can be found in– D. Lowe, Distinctive image featuresfrom scale‐invariant keypoints,IJCV 60(2), pp. 91‐110, 2004• Try the available local feature detectors and descriptors– http://www.robots.ox.ac.uk/~vgg/research/affine/detectors.html#binariesFei-Fei LiLecture 12 -765‐Nov‐11

What we have learned today• Local descriptor– SIFT (Problem Set 3 (Q2))– An assortment of other descriptors (Problem Set 3(Q4))• Recognition and matching with local features(Problem Set 3 (Q3))– Matching objects with local features: David Lowe– Panorama stitching: Brown & Lowe– ApplicationsFei-Fei LiLecture 12 -775‐Nov‐11

Supplementary materials• Matching images– Affine Transformation– Projective Transformation (Homography)Fei-Fei LiLecture 12 -785‐Nov‐11

Recognition with Local Features• Image content is transformed into local features that areinvariant to translation, rotation, and scale• Goal: Verify if they belong to a consistent configurationLocal Features,e.g. SIFTSlide credit: David LoweLecture 12 -79

Alignment Problem• We have previously considered how to fit a model to imageevidence– e.g., a line to edge points• In alignment, we will fit the parameters of sometransformation according to a set of matching feature pairs(“correspondences”).x iTx i'Slide credit: Kristen GraumanFei-Fei LiLecture 12 -805‐Nov‐11

Basic 2D Transformations• Basic 2D transformations as 3x3 matricesx' 10 txxy'0 1 t yy1 0 0 1 1Translationx' sx y' 01 00sy0Scaling0x0y11x' cossin0xy' sincos0y 1 0 0 1 1Rotationx ' 1 shx0xy ' shy1 0y 1 0 0 1 1ShearingSlide credit: Alyosha EfrosFei-Fei LiLecture 12 -815‐Nov‐11

What Can be Represented by a 22 Matrix?• 2D Scaling?x' s * xy' s * y• 2D Rotation around (0,0)?x' cos * xsin * yy' sin * xcos *y• 2D Shearing?xyx ' xsh * yy' sh * xyyxx' sx0 xy' 0 s y y x' cossinx y' sincosy x' 1 shxx y' shy1 y Slide credit: Alyosha EfrosFei-Fei LiLecture 12 -825‐Nov‐11

What Can be Represented by a 22 Matrix?• 2D Mirror about y axis?x ' xy' y• 2D Mirror over (0,0)?x 'xy'y• 2D Translation?x ' xty' ytxyx' 1 0xy' 0 1y x' 1 0 xy' 0 1y NO!Slide credit: Alyosha EfrosFei-Fei LiLecture 12 -835‐Nov‐11

2D Linear Transformsx ' a bxy' c dy • Only linear 2D transformations can berepresented with a 2x2 matrix.• Linear transformations are combinations of …– Scale,– Rotation,– Shear, and– MirrorSlide credit: Alyosha EfrosFei-Fei LiLecture 12 -845‐Nov‐11

Homogeneous Coordinates• Q: How can we represent translation as a 3x3 matrixusing homogeneous coordinates?x' x ty' y t• A: Using the rightmost column:xy1 0 txTranslation 0 1 ty0 0 1Slide credit: Alyosha EfrosFei-Fei LiLecture 12 -855‐Nov‐11

2D Affine Transformationx' ay' d w0be0c xf y1 w• Affine transformations:– Linear transformations– Translations• Parallel lines remain parallelProjective Transformation x' a y' d w' gbehc xf yi w• Projective transformations:– Affine transformations, and– Projective warps• Parallel lines do notnecessarily remain parallelFei-Fei LiLecture 12 -865‐Nov‐11

Let’s Start with Affine Transformations• Simple fitting procedure (linear least squares)• Approximates viewpoint changes for roughly planar objectsand roughly orthographic cameras• Can be used to initialize fitting for more complex modelsSlide credit: Svetlana Lazebnik, David LoweFei-Fei LiLecture 12 -875‐Nov‐11

Fitting an Affine Transformation• Assuming we know the correspondences, how do we get thetransformation?(i,yi)A 1A 2 A 3x x,y)B 3(i ixi yi mm13mm24xyiitt12xi 0yi0B 1B 20 0xiyi10mm0m1m t1t21234 xi yi Slide credit: Bastian LeibeFei-Fei LiLecture 12 -885‐Nov‐11

Fitting an Affine Transformation• Assuming we know the correspondences, how do we get thetransformation?m1 m2xm1m2x tii 1 xiyi0 0 1 0m3xi yi m3m4yi t2 0 0 x iyi0 1 m4yi t1 t2• How many matches (correspondence pairs) do we need to solve for thetransformation parameters?• Once we have solved for the parameters, how do we compute thecoordinates of the corresponding point for x , y new) ?(newSlide credit: Bastian LeibeFei-Fei LiLecture 12 -895‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• A projective transform is a mapping between any twoperspective projections with the same center of projection.– I.e. two planes in 3D along the same sight ray• Properties– Rectangle should map to arbitrary quadrilateral– Parallel lines aren’t– but must preserve straight lines• This is called a homographywx'wy' w p’******H* x* y* 1pPP1PP2Slide credit: Alyosha EfrosFei-Fei LiLecture 12 -905‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• A projective transform is a mapping between any twoperspective projections with the same center of projection.– I.e. two planes in 3D along the same sight ray• Properties– Rectangle should map to arbitrary quadrilateral– Parallel lines aren’t– but must preserve straight lines• This is called a homographywx' h11 h12 h13xwy' h21 h22 h23y w h31 h32 1 1p’HpSet scale factor to 1 8 parameters left.Slide credit: Alyosha EfrosFei-Fei LiLecture 12 -915‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• Estimating the transformationA 1B 3A 2 A 3B 1B 2Homogenous coordinatesImage coordinatesxxxAAA1 B 1x2 B 2xx3 B 3x' y'z'hhh112131hhh122232hh13231 x y1x x' 1 y y'z'1 z'Matrix notationx'x''Hx1x'z 'Slide credit: Krystian MikolajczykFei-Fei LiLecture 12 -925‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• Estimating the transformationA 1B 3A 2 A 3B 1B 2Homogenous coordinatesImage coordinatesxxxAAA1 B 1x2 B 2xx3 B 3x' y'z'hhh112131hhh122232hh13231 x y1x x' 1 y y'z'1 z'Matrix notationx'x''Hx1x'z 'Slide credit: Krystian MikolajczykFei-Fei LiLecture 12 -935‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• Estimating the transformationA 1B 3A 2 A 3B 1B 2xxxAAA1 B 1x2 B 2xx3 B 3Fei-Fei LiHomogenous coordinatesx' h y' hz'hxA1h11h11213131xBx1Bhhh h112223212 hy32hh1By13231B h1 x y1131Image coordinatesx x' 1 y y'z'1 z'Lecture 12 -Matrix notationx'x''Hx1x'z 'Slide credit: Krystian Mikolajczyk945‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• Estimating the transformationA 1B 3A 2 A 3B 1B 2Homogenous coordinatesImage coordinatesxxxAAA1 B 1x2 B 2xx3 B 3x' h y' hz'hxA1h11h11213131xBx1Bhhh h112223212 hy32hh1By13231B h1 x y1131yA1x x' 1 yy'z'1 z'h21xB h1 22yB h1h x h y 131B132B123Matrix notationx'x''Hx1z 'x'Fei-Fei LiLecture 12 -955‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• Estimating the transformationA 1B 3A 2 A 3Image coordinatesB 1B 2xxxAAA1 B 1x2 B 2xx3 B 3xAHomogenous coordinates1h11h31xBx1B h112 hy32By1B h1131A121h31Bx1B h122 h1 1 1 1 1 1 1yhxy32By1B hx h x x h y x h x h y h1231A 31 B A 32 B A 11 B 12 B 13Slide credit: Krystian MikolajczykFei-Fei LiLecture 12 -965‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• Estimating the transformationA 1B 3xxxAAA1 B 1x2 B 2xx3 B 3xhA 2 A 3B 1B 2Homogenous coordinates Image coordinatesh11xB h1 12yB h1 13h21xB h1 22yB h1y 1A1h x h y 1h x h y 1A31B132B1x h x x h y x h x h y hhA 31 B A 32 B A 11 B 12 B 131 1 1 1 1 1 111xB h y h x h x x h yB xA1 12 B113 A131 B1A13221xB h y h y h x y h yB yA1 22 B123 A131 B1A13231B13211B1231100Slide credit: Krystian MikolajczykFei-Fei LiLecture 12 -975‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• Estimating the transformationA 1B 3hh11xB h y h x h x x h yB xA1 12 B113 A131 B1A13221xB h y h y h x y h yB yA1 22 B123 A131 B1A132111100A 2 A 3B 1B 2xxxAAA1 B 1x2 B 2xx3 B 3xB0 . . .1yB0...110...x0B...1y0B.101. x yA. . .. . ..1A1xxBBAh 011 x yA1A.1..yyBB11 x y...A1A1hh h h h h hh 1111213212223313200 . . . Slide credit: Krystian MikolajczykFei-Fei LiLecture 12 -985‐Nov‐11

Fitting a Projective Transformation(i.e. Homography)• Estimating the transformation• Solution:xxxAAA– Null‐space vector of A– Corresponds to smallesteigenvector1 B 1x2 B 2xx3 B 3AhSVDUDVTv19 , ,vv99d Ud99Ah 01191ddA 1A 2 A 31999vv1191vv1999Minimizes least square errorTB 3B 1B 2Slide credit: Krystian MikolajczykFei-Fei LiLecture 12 -995‐Nov‐11

Uses: Analyzing Patterns and Shapes• What is the shape of the b/w floor pattern?Slide credit: Antonio CriminisiThe floor (enlarged)Fei-Fei LiLecture 12 -1005‐Nov‐11

Analyzing Patterns and ShapesSlide credit: Antonio CriminisiAutomatic rectificationFrom Martin Kemp The Science of Art(manual reconstruction)Fei-Fei LiLecture 12 -1015‐Nov‐11