Multiple View Geometry of Non-planar Algebraic Curves

¢¡Multiple View Geometry of Non-planar Algebraic CurvesJ. Yermiyahu Kaminski ¡ , Michael Fryers ¡ , Amnon Shashua ¢ and Mina Teicher ¡Bar-Ilan UniversityMathematics and Computer Science DepartmentRamat-Gan, IsraelThe Hebrew University of JerusalemSchool of Computer Science and EngeneeringJerusalem, IsraelAbstractWe introduce a number of new results in the context ofmulti-view geometry from general algebraic curves. Westart with the derivation of the extended Kruppa’s equationswhich are responsible for describing the epipolar constraintof two projections of a general (non-planar) algebraiccurve. As part of the derivation of those constraintswe address the issue of dimension analysis and as a resultestablish the minimal number of algebraic curves requiredfor a solution of the epipolar geometry as a function of theirdegree and genus.We then establish new results on the reconstruction ofgeneral algebraic curves from multiple views. We addressthree different representations of curves: (i) the regularpoint representation for which we show that the reconstructionfrom two views of a curve of degree admits two solutions,one of £degree and the other of degree ,(ii) the dual space representation (tangents) for which wederive a lower bound for the number of views necessary for£ £¥¤¦£¨§©reconstruction as a function of the curve degree and genus,and (iii) a new representation (to computer vision) basedon the set of lines meeting the curve which does not requireany curve fitting in image space, for which we also derivelower bounds for the number of views necessary for reconstructionas a function of the curve degree alone.1. IntroductionA large body of research has been devoted to the problemof analyzing the 3D structure of a scene from multipleviews. The multi-view theory is by now well understoodwhen the scene consists of point and line features — a sum-Part of this work was done while J.Y.K was at the Hebrew Universitypursuing his doctoral studies.mary of the past decade of work in this area can be found in[13] and references to earlier work in [6].The theory is somewhat fragmented when it comes tocurve features, especially non-planar algebraic curves ofgeneral degree. Given known projection matrices [23, 18,19] show how to recover the 3D position of a conic sectionfrom two and three views, and [24] show how to recover thehomography matrix of the conic plane, and [11, 25] showhow to recover a quadric surface from projections of its occludingconics.Reconstruction of higher-order curves were addressedin [16, 3, 21, 22]. In [3] the matching curves are representedparametrically where the goal is to find a reparameterizationof each matching curve such that in thenew parameterization the points traced on each curve arematching points. The optimization is over a discrete parameterization,thus, for a planar algebraic curve of degree, which is ¤ represented by points, one¤would need minimal number of parameters to solve forin a non-linear bundle adjustment machinery — with someprior knowledge of a good initial guess. In [21, 22] thereconstruction is done under infinitesimal motion assumptionwith the computation of spatio-temporal derivativesthat minimize a set of non-linear equations at many differentpoints along the curve. In [16] only planar algebraic curveswere considered.The literature is somewhat sparse on the problem of recoveringthe camera projection matrices from matching projectionsof algebraic curves.For example, [15, 16] show howto recover the fundamental matrix from matching conicswith the result that 4 matching conics are minimally necessaryfor a unique solution. [16] generalize this result tohigher order curves, but consider only planar curves.In this paper we address the general issue of multi-viewgeometry of (non-planar) algebraic curves starting with thederivation of the extended Kruppa’s equations which are re-1

,%¤#¤%sponsible for describing the epipolar constraint of two projectionsof a general (non-planar) algebraic curve. As partof the derivation of those constraints we address the issueof dimension analysis and as a result establish the minimalnumber of algebraic curves required for a solution of theepipolar geometry as a function of their degree and genus.On the reconstruction front, we address three differentrepresentations of curves: (i) the regular point representationfor which we show that the reconstruction from twoviews of a curve of degree admits two solutions, one of£degree and the other of degree , (ii) dual spacerepresentation (image measurements are tangent lines) forwhich we derive a formula for the minimal number of views££¥¤¦£¨§©necessary for reconstruction as a function of the curve degreeand genus, and (iii) a new representation (with regardto computer vision) based on the set of lines meeting thecurve which does not require any curve fitting in imagespace, for which we also derive formulas for the minimalnumber of views necessary for reconstruction as a functionof curve degree alone.2. Recovering the epipolar geometry fromcurve correspondencesAn algebraic relation between matching algebraic curvescan be regarded as an extension of Kruppa’s equations. Intheir original form, these equations have been introduced tocompute the camera-intrinsic parameters from the projectionof the absolute conic onto the two image planes [20].However it is obvious that they still hold if one replacesthe absolute conic by any conic, or even any planar algebraiccurve, which plane does not meet any of the cameracenters [16]. We will show next that a generalization forarbitrary algebraic (non-planar) curves is possible and is astep toward the recovery of epipolar geometry from matchingcurves.Let be a smooth irreducible (non-planar) curve ¡£¢ in ,whose degree is . Let ¨ be the camera matrix, © thecamera center, £¥¤§¦ the retinal plane and the image curve.Here we mention a short list of well known facts (see [14,12, 5, 4]):1. We recall that a singularity of a planar curve is simplya point where the curve admits more than one tangent.The curve will always contain singularities.2. For a generic position of the camera center, the onlysingularities of will be nodes, that is points with twodistinct tangents.3. We define the class of the planar curve to be the degreeof its dual curve. Let be the class of . It is a wellknownfact that is constant for a generic position ofthe camera center.4. For a generic position of © , will have same degreeand genus asNote it is possible to give a formula for as a functionof the degree and the genus of . According to Plückerformula, we have:££¥¤¦£ § © §¦ ¤§ © ¤¦£ §¦ ¤¦£¦nodes nodes §¤where nodes denotes the number of nodes of (note thatthis includes complex nodes). Hence the genus, the degreeand the class are related by:§¦2.1. Extended Kruppa’s equations¦ £ ¦We are ready now to investigate the recovery of©¦the,be the fundamentalepipolar geometry from matching curves. ¨ Let ,be the camera matrices. Let andmatrix and the first epipole, "! . We will need to considerthe two following #$&%(') +* % mappings:and. Both are defined on the first image plane;$-%('./%associates a point to its epipolar line in the first image,#while sends it to its epipolar line in the second image.Let , and be the image curves (projections of ontothe image planes). 0Let 0 and be the polynomials thatrepresent respectively and . 21 Let 21 and denote thedual curves, whose polynomials are 3respectively 3 and .Roughly speaking, the extended Kruppa’s equations statethat the sets of epipolar lines tangent to the curve in each imageare projectively related. A similar observation has beenmade in [2] for epipolar lines tangent to apparent contoursof objects, but it was used within an optimization scheme.Here we are looking for closed-form solutions, where noinitial knowledge of the answer is required. In order to developsuch a closed-form solution for the computation of theepipolar geometry, we need a more quantitative approach,which is given by the following theorem:Theorem 1 Extended Kruppa’s equationsFor a generic position of the camera centers with respectto the curve in space, there exists a non-zero scalar 4 , suchthat for all points % in the first image, the following equalityholds:3 ¤5, ¤ "463Observe that if is a conic 7 and and 7 the matricesthat respectively represent and , the extendedKruppa’s equations reduce to the classical Kruppa’s equa-9;:< 9 = :721 8 tions, that 721 is: 721 8 , whereare the adjoint matrices of 7and 7 . (1)and 721Proof: Let ? be the set of epipolar lines tangent to thecurve in image . We start by proving the following lemma.2

¤¤¤¦#¤¤¡¤¤§¦©©:¤¤¤Lemma 1 The two ? sets and ? are projectively equivalent.Furthermore for each corresponding pair ofepipolarlines, ¡£ ? ¦? , the multiplicity of and as points§£of the dual curves ¤¡ ¢ and ¥¤ 1 are the same. 1 Proof: Consider the three following pencils: = ¤¡© ¡ , the pencils of planes containing the base-¨ 1.line, generated by the camera centers,¨ =2. ¡ , the pencil of epipolar lines through thefirst epipole,¨ 3.second epipole. = ¡ , the pencil of epipolar lines through theThus ?¨ we have . Moreover if is the set of¤¡©planein tangent to the curve in space, there exist a one-toonemapping from ? to each . This mapping also leaves¨the multiplicities unchanged. This completes the lemma.This lemma implies that both side of the equation 1 definethe same algebraic set, that the union of epipolar linesthrough tangent to . 3Since 3 and , in the genericcase, have same degree (as stated in 2), each side of equation1 can be factorized into linear factors , satisfying thefollowing:As a result, in a generic situation every solution of'&$%+ 0¡$% ¦ ¦ '&¡#" : 2. 1.is admissible. Let be the sub-variety ofdefined by the equations togetherwith and , where is thesecond epipole. We have the two following results, whichare proven in [1]. The first of them gives a lower bound ofthe dimension 0 of as a function of , whereas the secondgives a sufficient condition for 0 a to be a finite set.Proposition 2 If 0 is non-empty, the dimension of 0 is at3 least .Proposition 3 For a generic position of the camera centers,the 0 variety will be discrete if, for any pointis not contained in any quadric surface. ¥¤0 , the union of © and the points 465Observe that these two results are consistent, since therealways exist a quadric surface containing a given line andsix given points. However in general there is no quadriccontaining a given line and seven given points. Thereforewe can conclude with the following theorem.Theorem 2 For a generic position of the camera centers,the extended Kruppa’s equations define the epipolar geometryup to a finite-fold ambiguity if and only 3 if . 3¤ ¢ 3 ¤5, ¤ ¢ ¤¡¤¡4 ¢ ¢ ( where . By the previous lemma, wemust also ! have for .By eliminating the 4 scalar from the extended Kruppa’sequations (1) we obtain a set of bi-homogeneous equationsin and . Hence they define a variety ¡ ¦ ¡#" in . Thisgives rise to an important question. How many of thoseequations are algebraically independent, or in other wordswhat is the dimension of the set of solutions? This is theissue of the next section.Since different curves in generic position give rise to independentequations, this result means that the sum of theclasses of the image curves must be at 3 least 0 for to be afinite set. Observe that this result is consistent with the factthat four ( conics for each conic) in general positionare sufficient to compute the fundamental matrix, as shownin [15, 16]. Now we proceed to translate the result in termsof the geometric properties of directly using the degreeand the genus of , related to by the following¦ £relation:¦ § ¦. Here are some examples for sets of curvesthat allow the recovery of the fundamental matrix:1. Four conics ( ¦ ! ) in general position.2. Two rational cubics ( £ ! ) in general position.£2.2. Dimension of the set of solutions$%+'&Let be the set of bi-homogeneous equations on and , extracted from the extended Kruppa’sequations (1). Our first concern is to determine whether allsolutions of equation (1) are admissible, that is whether theysatisfy the usual ! constraint . Indeed the followingstatement can be proven [1]:Proposition 1 As long as there are at least 2 distinctlines through tangent to , equation (1) implies that(')+*-, and /. .3. A rational cubic and two conics in general position.4. Two elliptic cubics ( £(see also [16]).5. A general rational quartic ( 87£ 7 ! ), and a generalelliptic quartic ( £3. 3D Reconstruction).) in general positionWe turn our attention to the problem of reconstructing analgebraic curve from two or more views, given known cameramatrices (epipolar geometries are known). The basic3

¤&9¨¢:¤¤¤¢¢idea is to intersect together the cones defined by the cameracenters and the image curves. However this intersection canbe computed in three different spaces, giving rise to differentalgorithms and applications. Given the representation inone of those spaces, it is possible to compute the two otherrepresentations [1].We shall mention that in [10] a scheme is proposed to reconstructan algebraic curve from a single view by blowingupthe projection. This approach results in a spatial curvedefined up to an unknown projective transformation. In factthe only computation this reconstruction allows is the recoveryof the projective properties of the curve. Moreover thisreconstruction is valid for irreducible curves only. Howeverreconstructing from two projections not only gives the projectiveproperties of the curve, but also the relative depth ofit with respect to others objects in the scene and furthermorethe relative position between irreducible components.3.1. Reconstruction in Point SpaceLet the camera projection 8 ¢¡'.matrices 8 £¤¡ 9be and ,§¦¥ §©¨ §©¨ ¨ where , see [17]. Hence the two cones definedby the image curves and the camera centers are given£¤ 9 by: ¤ 0 ¤8 £¤¡ 9 and . Thereconstruction is defined as the curve whose equations are ! and "08 ¢¡'. ! . The irreducible components of the intersection(the separate curves) have the following degrees:Theorem 3 For a generic position of the camera centers,that is when no epipolar plane is tangent twice to the curve, the curve defined $¢ ! !by has two irreduciblecomponents. One has degree and is the actualsolution of the reconstruction. The other one has degree£ § © £¥¤¦£.We skip the proof because of its technical content. Thereader could find it in [1]. This result provides an algorithmto find the right solution for the reconstruction in a genericconfiguration, except in the case of conics, where the twocomponents of the reconstruction are both admissible.3.2. Reconstruction in the Dual Space1 Let be the dual variety of , that is, the set of planestangent to . Since is supposed not to be a line, the dualvariety 1 must be a hypersurface of the dual space [12].Hence let be a minimal degree polynomial that represents1 . Our first concern is to determine the degree of .Proposition 4 The degree of is , that is, the commondegree of the dual image curves.Proof: Since 1 is a hypersurface of ¡¢ 1 , its degree is thenumber of points where a generic line in ¡£¢ 1 meets 1 .By duality it is the number of planes in a generic pencilthat are tangent to . Hence it is the degree of the dualimage curve. Another way to express the same fact is theobservation that the dual image curve is the intersection of1 with a generic plane in ¡ ¢ 1 . Note that this provides anew proof that the degree of the dual image curve is constantfor a generic position of the camera center.For the reconstruction ¦ of from multiple view, we willneed to consider the mapping from a line of the imageplane to the plane that it defines with the camera center. Let$ ' denote this mapping [7]. There exists a link involving , and 3 , the polynomial of the dual imagecurve: ¤ ¤¡ ¤¡ ! . Since these twopolynomials have the same degree (because is linear) and ! whenever 33 is irreducible, there exist a scalar 4 such thatfor all lines ¤ ¡linear equations on is¤ ¤¡ "463¤¡ §©. Since the number of coefficients in, we can state the following result:1 . Eliminating 4 , we getProposition 5 The reconstruction in the dual space can bedone linearly using at ¤ ¨ least views.for few ex-The lower bounds on the number of viewsamples are given below:¤ ¦¤ 7 7 7¤ ¤¡ ! § © 1. for a conic locus, ,2. for a rational cubic, ,3. for an elliptic cubic, ,4. for a rational quartic, ,5. for a elliptic quartic, .Moreover it is worth noting that the fitting of the dualimage curve is not necessary. It is sufficient to extracttangents to the image curves at distinct points. Each tangentcontributes to one linear equation on : .However one cannot obtain more thanlinearly independent equations per view.3.3. Reconstruction in ¤ © As a third representation of the curve , we considerthe set of lines meeting . This defines completely thecurve , as shown in [12]. As we shall see, we will paythe price of requiring extra views for reconstruction but willgain from the fact that we can use the image points directlywithout the need to perform curve fitting.A line can either be represented by a pair of planes ¤ ¤¡ ¢©¦ ¡ ¢© that contain it or by its Plücker coordinatesin the Grassmannian © ¤. For the purpose of4

¤©¤%¤¤©¤%¤©¤¤¤%¤¡¤¤%¦¡¤£¤%!¦¡¤¦¡¨!!!¨!©!!!!computation with curves, we found that the representationby Plücker coordinates is more convenient.Hence the curve of degree can be represented by ahomogeneous polynomial , called in that context the Chowpolynomial of , of £ degree , that defines a hypersurface£ ¤ © in . Note that is not uniquely defined. Two suchpolynomials must differ by a multiple of the quadratic¤equationdefining as a ¡¢¡ sub-variety of . However pick-© ing one representative of this equivalence class is sufficientto reconstruct entirely without any ambiguity the curve[12]. The number of coefficients £¥¤ in is . Howeversince it is defined modulo the quadratic¤ ©equation definingindependentlinear conditions on its coefficients to determine one, it is sufficient to provide £ ¤¤¨§ -£¤¨§instance of .0 Let be the polynomial defining the image curve, .© $ %' ¨ % ¨ ¦ ¨ 00Consider the mapping that associates to an image point itsoptical ray: , where is a matrix, whichentries are polynomials functions of [7]. Hence the polynomialvanishes whenever does. Since theyhave same degree and is irreducible, there exists a scalar, we have:1110.5z109.59–1–0.5y00.500.2x 0.40.60.81 1Figure 1. A spatial quarticFigure 2. An electric thread.The curve is smooth and irreducible, and has 7 degreeand genus . We define two camera matrices:©4 such as for every point %©§ © ¦ § ©This yields linear equations on .£ ¤Hence a similar statement to that in Proposition 5 can bemade:Proposition 6 The ¨ reconstructioninlinearly using at least¤ ¡ ¤¤ "© ¤can be doneviews.For some examples, below are the minimal number of¤viewsfor a linear reconstruction of the © curve in :1. for 7 a conic locus, ,2. ¤ for a cubic, ,¤3. for a quartic, .As in the case of reconstruction in the dual space, itis not necessary 0 to explicitly compute . It is enough topick points on the image curve. Each point yieldsalinear! equation on :. However for each£view, onecannot ¢extract more than independent linearequations.4. ExperimentsWe start with a synthetic experiment followed later by areal image one. Consider the curve , drawn in figure 1,defined by the following equations: ¢ ¤ ¤ ¢ § ! !! !§ ©§ ¦ ©© § ! !! !The reconstruction of the curve from the two projectionshas been made in the point space, using FGb 1 , a powerfulsoftware tool for Gröbner basis computation [8, 9]. Asexpected there are two irreducible components. One has degreeand is the original curve, while the second has 7 degree© ¦.For the next experiment, we consider seven images of anelectric wire — one of the views is shown in figure 2. Ineach image, we perform segmentation and thinning of theimage curve. Hence for each of the images, we extracted aset of points lying on the thread. No fitting is performed inthe image space. For each image, the camera matrix is calculatedusing the calibration pattern. Then we proceeded tocompute the Chow polynomial of the curve in space. Thecurve has degree . Once is computed, a reprojectionis easily performed, as shown in figure 3.5. ConclusionIn this paper we have focused on general (non-planar) algebraiccurves as the building blocks from which the camerageometries are to be recovered and as the scene buildingblocks for the purpose of reconstruction from multipleviews. The new results derived in this paper include:1 Logiciel conçu et réalisé au laboratoire LIP6 de l’université Pierre etMarie CURIE.§ ©§ © © 460 §¤¡ § ©5

Figure 3. Reprojection on a new image.1. Extended Kruppa’s equations for the recovery ofepipolar geometry from two projections of algebraiccurves.2. Dimension analysis for the minimal number of algebraiccurves required for a solution of the epipolar geometry.3. Reconstruction from two views of a curve of degree £is a curve which contains two irreducible componentsone of degree £ and the other of degree £¥¤¦£ § © — aresult that leads to a unique reconstruction of the originalcurve.4. Formula for the minimal number of views required forthe reconstruction of the dual curve.5. Formula for the minimal number of views required¤forthe reconstruction of the © curve representation .AcknowledgmentWe express our gratitude to Jean-Charles Faugere for giving usaccess to his powerful system FGb. This research is partially supportedby the Emmy Noether Research Institute for Mathematicsand the Minerva Foundation of Germany and the excellency center(Group Theoretic Method In The Study Of Algebraic Varieties) ofthe Israeli Academy of Science.References[1] J.Y.Kaminski Multiple-view Geometry of AlgebraicCurves. Phd dissertation, submitted to Hebrew University ofJerusalem, May. 2001.[2] K. Astrom and F. Kahl. Motion Estimation in Image SequencesUsing the Deformation of Apparent Contours. InIEEE Transactions on Pattern Analysis and Machine Intelligence,21(2), February 1999.[3] R. Berthilson, K. Astrom and A. Heyden. Reconstruction ofcurves in ¢¡ , using Factorization and Bundle Adjustment. InIEEE Transactions on Pattern Analysis and Machine Intelligence,21(2), February 1999.[4] D. Eisenbud. Commutative Algebra with a view toward algebraicgeometry. Springer-Verlag, 1995.[5] D. Eisenbud and J. Harris. The Geometry of Schemes.Springer-Verlag, 2000.[6] O.D. Faugeras Three-Dimensional Computer Vision, A geometricapproach. MIT Press, 1993.[7] 0.D. Faugeras and T. Papadopoulo. Grassman-Cayley algebrafor modeling systems of cameras and the algebraic equationsof the manifold of trifocal tensors. Technical Report - INRIA3225, July 1997.[8] J.C. Faugere. Computing Grobner basis without reduction tozero (£¥¤ ). Technical report, LIP6, 1998.[9] J.C. Faugere. A new efficient algorithm for computing Grobnerbasis (£§¦ ).[10] D. Forsyth, Recognizing algebraic surfaces from their outlines.[11] Quadric Reconstruction from Dual-Space Geometry, 1998.[12] J. Harris Algebraic Geometry, a first course. Springer-Verlag, 1992.[13] Multiple View Geometry in computer vision. CambridgeUniveristy Press, 2000.[14] R. Hartshorne. Algebraic Geometry. Springer-Verlag, 1977.[15] F. Kahl and A. Heyden. Using Conic Correspondence in TwoImages to Estimate the Epipolar Geometry. In Proceedings ofthe International Conference on Computer Vision, 1998.[16] J.Y. Kaminski and A. Shashua. On Calibration and Reconstructionfrom Planar Curves. In Proceedings European Conferenceon Computer Vision, 2000.[17] Q.T Luong and T. Vieville. Canonic Representations for theGeometries of Multiple Projective Views. In Proceedings EuropeanConference on Computer Vision, 1994.[18] S.D. Ma and X. Chen. Quadric Reconstruction from its OccludingContours. In Proceedings International Conferenceof Pattern Recognition, 1994.[19] S.D. Ma and L. Li. Ellipsoid Reconstruction from ThreePerspective Views. In Proceedings International Conferenceof Pattern Recognition, 1996.[20] S.J. Maybank and O.D. Faugeras A theory of self-calibrationof a moving camera. International Journal of Computer Vision,8(2):123–151, 1992.[21] T. Papadopoulo and O. Faugeras Computing structure andmotion of general 3d curves from monocular sequences ofperspective images. In Proceedings European Conference onComputer Vision, 1996.[22] T. Papadopoulo and O. Faugeras Computing structure andmotion of general 3d curves from monocular sequences of perspectiveimages. Technical Report 2765, INRIA, 1995.[23] L. Quan. Conic Reconstruction and Correspondence fromTwo Views. In IEEE Transactions on Pattern Analysis andMachine Intelligence, 18(2), February 1996.[24] C. Schmid and A. Zisserman. The Geometry and Matchingof Curves in Multiple Views. In Proceedings European Conferenceon Computer Vision, 1998.[25] A. Shashua and S. Toelg The Quadric Reference Surface:Theory and Applications. International Journal of ComputerVision, 23(2):185–198, 1997.6