Fingerprint Deformation Models Using Minutiae Locations and ...

Appeared in Proc. of IEEE Workshop on Applications of Computer Vision (WACV), (Breckenridge, Colorado), pp. 150-155, January 2005Fingerprint Deformation Models Using Minutiae Locations and OrientationsYi Chen, Sarat Dass, Arun Ross, and Anil JainDepartment of Computer Science and EngineeringMichigan State UniversityEast Lansing, MI, 48823AbstractNonlinear deformations in fingerprint images, arising fromthe elasticity of the skin as well as the pressure and movementof the finger during image acquisition, lead to difficultiesin establishing a match between multiple impressionsacquired from the same finger. One solution to this problemis to estimate and remove the relative deformationsprior to the matching stage. In this paper, these relativedeformations are represented as an average deformationmodel based on minutiae locations and orientations using2-D Thin Plate Splines (TPS). The estimated average deformationis used to pre-distort a template prior to matchingit with a query image in a verification task. Experimentalresults show that the use of minutiae locations and orientationsto estimate the deformation leads to a more representativedeformation model than using minutiae locationsonly. An index of deformation based on the bending energyis also proposed to select templates with the least variabilityin the deformations. The EER goes down by ≈ 1.1%when we incorporate minutiae orientation information anduse the template selection strategy.1 IntroductionThere are several factors that impact fingerprint deformation:the amount of pressure applied by the subject, the dispositionof the subject (sitting or standing), the motion ofthe finger, the moisture content of the skin (dry, oily or wet),the elasticity of the skin. Recently, several methods havebeen presented to deal with this deformation from differentviewpoints. Ratha et al. [3] proposed a controlled acquisitionprocess by measuring the forces and torques on thescanner directly with the aid of specialized hardware. Senioret al. [4] suggested an automatic method to remove deformationsby enforcing the constraint that ridges should beconstantly spaced, with deviations from constant spacingsindicating the presence of deformations. Watson et al. [12]constructed deformation filters for template fingerprint imagesprior to using a correlation-based matching. Kovács-Vajna [7] proposed a matching method that takes deforma-Figure 1: Three impressions of a single finger showing the effectof varying deformations.tions into account by employing tolerance bounds for interminutiaedistances and angles. Bazen et al. [5, 6] used aThin-Plate Spline (TPS) model to describe the non-lineardeformations between two minutiae sets. However, mostof these techniques deal with the problem of non-linear deformationon a case-by-case basis. No attempt was madeto develop a finger-specific deformation model that can becomputed offline and then used for matching. The main advantageof an offline technique is that once a finger-specificmodel has been computed, recomputation of the model isnot necessary during the matching stage. Also, by usingmultiple impressions of the same finger to compute deformationsfrom a template, we avoid estimating optimal deformationsfor impostor pairs, resulting in low FAR values.In [1], we have proposed such a finger-specific deformationmodel using 2-D TPS based on minutiae point patterns.The TPS, a spatial generalization of the cubic spline, is aneffective tool for estimating the deformation that warps oneset of point patterns to another. The TPS model in [1] iscomputed by distorting the minutiae set of one fingerprintimpression to match with others from the same finger. Althoughthe deformations observed in a fingerprint vary fromone acquisition to the next (Fig.1), it forms an informativeand representative feature space which is best characterizedthrough statistical modeling. However, this method onlyemployed the locations of minutiae points during deformationestimation, ignoring the fact that orientations of minutiaepoints also provide additional and useful informationfor a better estimate of the deformation.In this paper, both the orientation and location informationof minutiae are incorporated in the TPS model. A new

Appeared in Proc. of IEEE Workshop on Applications of Computer Vision (WACV), (Breckenridge, Colorado), pp. 150-155, January 2005method to establish minutiae correspondences is also developedto compensate for the sensitivity of the TPS model to estimated, where c is a 2 × 1 vector, A is a 2 × 2 matrix andA total of (6 + 2l) parameters in equation (2) need to beincorrect correspondences. Incorrect correspondences are W is a l×2 matrix. Since equation (1) enforces l constraintscaused due to false, missed or displaced minutiae points, or in the spatial domain, we can reduce the number of degreeswhen there is limited overlap between image pairs, resulting of freedom by 2l. Further, we assume that the coefficientsin an unreliable TPS model. We also introduce the use of W satisfy (i) 1 T l W = 0 (2 restrictions) and (ii) U T W = 0the variance of the “bending energy,” instead of pixel-wise (4 restrictions), where 1 l is a vector of ones of length l.covariance matrix of deformations as in [1], to rank the relativedeformations associated with a finger. We demonstrate from the matrix equation:Thus, the coefficients of the TPS model can be obtainedthat using the variance of the “bending energy” leads to⎡greater computational efficiency and a better performancecompared to [1].⎣ S 1 ⎤ ⎡l U1 T l 0 0 ⎦ ⎣ W ⎤ ⎡c T ⎦ = ⎣ V ⎤0 ⎦ , (4)The rest of the paper is organized as follows. SectionU T 0 0 A T 02 describes the general methodology, including the basicconcepts of the TPS model. Section 3 gives a modified algorithmfor obtaining reliable correspondences. Section 4 The above matrix gives rise to a TPS model that minimizeswhere S is the l×l matrix of (σ(u j −u k )), j, k = 1, 2, ..., l.describes the transformations needed for incorporating the the bending energy subject to the perfect alignment constraintsin equation (1). A more robust TPS model can beminutiae orientations in the TPS model. Section 5 developsan average deformation model for a template and ranks templatesof the same finger based on the variance of the “benddefiningthe transformation F which minimizes the expres-obtained by relaxing the constraints in equation (1) and reingenergy.” Experimental results are provided in Section 6, sionwith summary and future work presented in Section 7.l∑(v j − F (u j )) T (v i − F (u i )) + λJ(F ), (5)2 General MethodologyGiven a pair of grayscale fingerprint images, I 0 and I 1 , definedon a spatial domain R 2 , the deformation from I 0 to I 1is given by a deformation function F : R 2 → R 2 such thatF (I 0 ) = I 1 . In this paper, we focus on modelling F usingthe Thin-Plate Splines (TPS). Pioneered by Bookstein [8],the TPS modelling is particularly useful when two or threedimensionallandmarks in a reference need to be mapped tothe corresponding landmarks in a target such that the “bendingenergy” of F is minimized.Specifically, let U = (u 1 , u 2 , ..., u l ) T and V = (v 1 , v 2 ,..., u l ) T be a pair of point sets with known correspondences,derived from I 0 and I 1 , respectively. We assume U and Vhave been aligned using the Procrustes analysis [10]. Here,u k and v k denote the spatial coordinates of the kth correspondingpair and l is the total number of correspondences.The deformation function F is then required to satisfyF (u i ) = v i , i = 1, 2, ..., l. (1)The TPS estimate of F is given by parameter vectors c, Aand W :F (u) = c + A · u + W T s(u), (2)where u ∈ R 2 , c and A define the affine parts of thetransformation and W gives the additional non-linear deformation.The distance measure s(u) is the vector (σ(u −u 1 ), σ(u − u 2 ), ..., σ(u − u l )) T withσ(u) = ‖u‖ 2 log ‖u‖. (3)wherej=1J(F ) =2∑∫j=1{( ∂2 F j (x, y)R ∂x 2 ) 2 + 2( ∂2 F j (x, y)∂x∂y2+( ∂2 F j (x, y)∂y 2 ) 2 }dxdy (6)represents the bending energy associated with F =(F 1 , F 2 ) T , with F j as the j-th component of F , and λ > 0.The coefficients of this resulting TPS model can be obtainedusing equation (4) with S replaced by S + λI l , where I l isthe l × l identity matrix. Generally, these splines do not exactlyinterpolate all landmark points, but are allowed to approximatethem in favor of a smoothing parameter λ. Whenλ increases, the resulting spline becomes more smooth.3 Establishing Point CorrespondencesThe TPS model relies on correct correspondences of pointpatterns during deformation estimation. Thus, for fingerprintimages, the accuracy of finding minutiae correspondencescan greatly affect the efficacy and efficiency of theTPS. In order to obtain reliable minutiae correspondences,we modified the Elastic Point Pattern Matching (EPPM) algorithmproposed in [9] by incorporating a voting schemeto find the optimal global translation and rotation. The procedureis described as follows:) 2

Appeared in Proc. of IEEE Workshop on Applications of Computer Vision (WACV), (Breckenridge, Colorado), pp. 150-155, January 2005(a)(b)(c)(d)Figure 2: Improvement in correspondences (indicated by arrows) between two sets of minutiae point patterns. (a) and (b) are correspondencesobtained by the EPPM algorithm while (c) and (d) are correspondences obtained by the new algorithm for the same pairs offingerprint images.1. Given a pair of images, I 0 (called template) andI 1 (called query), we extract minutiae points usingthe method proposed in [9]. Assume M 0 =(u 1 , u 2 , ..., u m0 ) is the set of minutiae points extractedfrom I 0 and M 1 = (v 1 , v 2 , ..., v m1 ) from I 1 . Wealso trace the ridge associated with each minutiae pointand extract sample points on that ridge (sampled every5-th point), with R ui = (u ∗ i,1 , u∗ i,2 , ..., u∗ i,a ) andR vj = (vj,1 ∗ , v∗ j,2 , ..., v∗ j,b ) denoting the ridge pointscorresponding to the minutiae u i and v j , respectively.2. For a total of m 0 · m 1 possible pairings of minutiaepoints, we select one (u i , v j ), (i = 1, 2, ..., m 0 ,j = 1, 2, ..., m 1 ) as the reference pair and transformremaining minutiae points in M 0 and M 1 ,respectively,to polar coordinates with centers at u i and v j .3. The query minutiae set M 1 is rotated about its referenceminutiae v j in order to find candidate pairingswith M 0 . Each candidate pair is verified using abounding box. Let MP ij be the total number of matchingpairs for (u i , v j ).4. Compute (∆x ij , ∆y ij , ∆θ ij ) as the translational (in xand y directions) and rotational offsets associated with(u i , v j ). Then add votes to V ote(∆x ij , ∆y ij , ∆θ ij )with the number of votes equal to MP ij .5. Perform steps 2-4 until all possible reference pairs areconsidered and votes for the corresponding transformationparameters are obtained.6. Establish bins with size of (20, 20, 15) and collectvotes for each bin. The top 5 bins with themaximum number of votes are selected and each(∆x ij , ∆y ij , ∆θ ij ) that contribute to one of the top 5bins are sent to a 2-D dynamic matching algorithm toget the matching score [13].7. The one that results in the highest matching score is finallychosen as the optimal global transformation andis used to establish the individual minutiae correspondenceswithin a bounding box.Fig. 2 shows examples of correspondences generated byusing the original EPPM algorithm [9] and the modified algorithm.Let (T/C) denote the ratio of the detected correspondencesto the number of correct correspondences thatare manually verified. Figs. 2(c) and 2(d) give higher ratiosof (7/7) and (10/10) compared to Figs. 2(a) and 2(b) thatgive low values of (0/4) and (1/3).4 Incorporating Minutiae OrientationOne difficulty in incorporating minutiae orientation informationis that transformation in the orientation space isnot equivalent to transformation in the the spatial domain;for example, rotation in the spatial domain corresponds totranslation in the orientation space. Thus, in order to use theTPS to model deformations using minutiae locations andorientations, we convert the orientation information into thespatial domain using the following method.

Appeared in Proc. of IEEE Workshop on Applications of Computer Vision (WACV), (Breckenridge, Colorado), pp. 150-155, January 2005u c =(xc,y c)5.1 Index of Deformationv c =(xc',yc')In [1, 2], a method to rank the average deformation modelsusing the pixel-wise covariance matrix of the deformationsu=(x,y)(v=(x',y'))'was given. However, this method is computationallyexpensive and does not necessarily reflect non-linear deformationsbecause of its sensitivity to linear transformations.We propose a new ranking method based on the bendingenergy, a measure of non-linear deformations that the TPStries to minimize. The minimized bending energy can beFigure 3: Obtaining angle points u c and v c based on the minutiae computed byorientations θ and θ ′ . Note that u and v are aligned for simplicityof illustration.J(F ) = trace(W T SW ), (9)Let the location and orientation of a minutiae point bedenoted by u = (x, y) and θ, respectively. We draw aunit circle around u and find its corresponding “angle point”u c = (x c , y c ), where:x c = x + cos θ, y c = y + sin θ. (7)Assume u is in correspondence with the minutiae pointv = (x ′ , y ′ ). The angle point v c = (x c , y c ) of v is foundin the same way using the minutiae orientation θ ′ for v (seeFig. 3). Therefore, orientation information is converted intothe spatial domain as we generate an angle point pair foreach corresponding minutiae pair. As a result, the numberof corresponding point patterns for the TPS model is doubled.The TPS deformation model is found as in Section2 with U and V now containing both minutiae points andangle points.5 Average Deformation ModelsSuppose we have N impressions of a finger, I 1 , I 2 , ..., I N .Each impression I k can be paired with the remaining impressionsto create N − 1 pairs of the form (I i , I j ), i ≠ j.For the pair (I i , I j ), we obtain a non-linear deformationmodel F ij by employing the TPS technique described inSection 2. Note that F ij then transforms every pixel in thetemplate fingerprint I i , to a new location. Thus, we cancompute the average deformation of each pixel u in I i as:¯F i (u) = 1N − 1N∑F ij (u). (8)j=1j≠iThe average deformation is the typical deformation thatarises when we compare one impression of a finger (thebaseline impression) with others of the same finger. Thisoffline process is repeated for every impression I i , i =1, 2, .., N, resulting in N average deformation models thatcan be incorporated in the matching stage.where W is the parameter for the non-linear transformationand S is the distance matrix (see equation (4)).Given N impressions per finger, if we choose I i as thetemplate image, N − 1 deformation functions F ij will beobtained when I i is compared to I j , j = 1, 2, ..., N, j ≠ i.Thus, a total of N − 1 values of bending energy J(F ), forF = F ij , j = 1, 2, ..., N, j ≠ i, are provided. The Φ-indexof deformation for template I i is then defined as:Φ i = 1N − 1N∑(J(F ij ) − ¯J i ) 2 , (10)j=1j≠iwhere ¯Ji = 1N − 1N∑J(F ij ). (11)j=1j≠iLow (high) values of the Φ-index indicate that theamount of bending energy applied by each deformationfunction is similar (dissimilar) to each other. However, lowvalues of the Φ-index also occurs when the number of correspondencesis consistently small between a template andother impressions of the same finger, which should not beconsidered as a potentially good template. Therefore, foreach template I i , we compute its total number of correspondencesC i with each I j , j = 1, 2, ..., N, j ≠ i andthe mean of total correspondences for impressions of a fingerby ¯C = 1 ∑N Ci , i = 1, 2, ..., N. We exclude thosetemplates with C i < ¯C, i = 1, 2, ..., N, during the ranking.Among the remaining templates, the one I i ∗ withi ∗ = argmin ¯CΦ i:Ci≥ i , is the optimal template for a finger.It is considered to have the smallest variability in the nonlineardeformations compared to all other templates (Fig.4).6 Experimental ResultsIn order to apply the TPS model to reliably estimate fingerprintdeformation, we need to have several impressionsof the same finger (at least 10). Large number of impressionsof a finger are not available in standard fingerprint

Appeared in Proc. of IEEE Workshop on Applications of Computer Vision (WACV), (Breckenridge, Colorado), pp. 150-155, January 2005(a)Φ = 0.0030(b)Φ = 0.0151(c)Φ = 0.0242(d)Φ = 0.1061Figure 4: Four templates before (left) and after (right) applying the average deformation model based on minutiae locations and orientationsare shown on a reference grid with minutiaes marked. They correspond to (a) the highest Φ-value (b) the 6-th highest Φ-value (c) the11-th highest Φ-value and (d) the lowest Φ-value when ranked using the bending energy.databases (e.g., FVC 2002 [11]) and therefore, we used datacollected in our lab. We acquired fingerprint images of 50fingers using the Identix Bio Touch USB 200 optical sensor(256×255, 380 dpi) over a period of two weeks. There were32 impressions corresponding to every finger, resulting in atotal of 1600 impressions. One half of the impressions (16impressions per finger, with a total of 800 impressions) wereused as template to compute the deformation model whilethe remaining 800 impressions were used as query imagesfor testing. The minutiae points and their correspondingorientation information were extracted using the algorithmoutlined in [9]. For the 16 test impressions per finger, thedeformations F ij based on minutiae locations and orientationswere obtained by fixing I i as the template and the remainingI j , j ≠ i as the query with λ = 5 in equation (5).The average deformation model F i was then computed usingequation (8). During the matching stage, M i , the minutiaeset of I i , was deformed to MD i = ¯F i (M i ). Thus, atotal of 800 sets (50 × 16) of average deformation modelswere obtained. We also use MDi ∗ to denote the deformedminutiae set using the TPS model based only on minutiaelocations [1].In the first experiment, every template image I i was comparedwith every query image I j . A modified matcher basedon the 2-D dynamic matching algorithm in [13] was usedto generate three types of matching scores for each comparison:the matching score obtained by matching (i) M iwith M j (no adjustment for deformations), (ii) MD ∗ i withM j (deformed based on minutiae locations only) and (iii)MD i with M j (deformed based on minutiae locations andorientations). The Receiver Operating characteristic (ROC)curve plotting the genuine accept rate (GAR) against thefalse accept rate (FAR) at various thresholds is presentedin Fig. 5(a). When the average deformation model basedon minutiae locations is applied to pre-distort the templateprior to matching, an improvement of approximately 3.0%with respect to the non-deformed templates is observed ata false accept rate of 0.01%. Using minutiae locations andorientations results in an additional ≈ 1.0% improvement.In the second experiment, the advantage of using the Φ-index is demonstrated. We generate two sets of optimaltemplates using (a) the knowledge of average pixel-wisedeformation and (b) the knowledge of the bending energy.Each set contains 50 optimal templates (one per finger) fromthe training data and 800 queries from the test data. Fig.5(b) shows the ROC curve by applying these two optimalsets, together with the ROC curve using method (i) in thefirst experiment. We can see that the ranking method basedon the bending energy not only provides computational efficiency,but also leads to better performance in matching.7 Summary and ConclusionsWe have proposed a method to generate average deformationmodels based on minutiae locations and orientations.By utilizing minutiae orientations, the proposed method improvesthe performance of a fingerprint matching system. Amodified correspondence algorithm is also proposed to gen-

Appeared in Proc. of IEEE Workshop on Applications of Computer Vision (WACV), (Breckenridge, Colorado), pp. 150-155, January 200595959090Genuine Accept Rate(%)85Genuine Accept Rate(%)858080Without DeformationAverage Warping Using Minutiae LocationsAverage Warping Using Minutiae Locations and Orientations7510 −2 False Accept Rate(%)10 −1Average Warping Using Minutiae Locations and OrientationsUsing Optimal Templates Ranked by Pixelwise DeformationUsing Optimal Templates Ranked by Bending Energy7510 −2 False Accept Rate(%)10 −1(a)Figure 5: (a) Improvement in matching performance using non-deformed templates (‘×’), deformed templates using the average deformationmodel based on: minutiae locations (‘∇’) and minutiae locations and orientations (‘◦’). (b) Improvement in matching performanceusing all deformed templates (‘◦’), the optimal templates ranked by Φ-index based on: pixel-wise covariance of deformation (‘- -’) andbending energy (‘...’) when the average deformation model based on minutiae locations and orientations is applied.(b)erate reliable correspondences prior to computing the TPSmodel. In addition, a more efficient and robust measure togenerate the index of deformation is defined by using thebending energy of deformations. Future work includes incorporatingmore features like ridges and ridge orientationsinto the framework. We also plan to adopt an incrementalapproach to update the average deformation model in acoarse-to-fine fashion.References[1] A. Ross, S.C. Dass and A.K. Jain, “A Deformable Model forFingerprint Matching,” Pattern Recognition, Vol.38(1), pp.95-103, 2005.[2] A. Ross, S.C. Dass and A.K. Jain, “Estimating FingerprintDeformation,” Proceedings of the International Conferenceon Biometric Authentication (ICBA), Vol. 3072, pp. 249-255, Hong Kong, 2004.[3] N.K. Ratha and R.M. Bolle, “Effect of controlled image acquisitionon fingerprint matching,” Proceedings of the 14thIAPR International Conference on Pattern Recognition, Vol.2, pp. 1659-1661, Brisbane, 1998.[4] A. Senior and R. Bolle, “Improved fingerprint matching bydistortion removal,” IEICE Transactions on Information andSystem, Special issue on Biometrics, Vol. E84-D(7), pp. 825-831, 2001.[5] A.M. Bazen and S.H. Gerez, “Elastic minutiae matchingby means of thin-plate spline models,” Proceedings of16th ICPR International Conference on Pattern Recognition,Québec City, Canada, 2002.[6] A.M. Bazen and S.H. Gerez, “Fingerprint matching bythin-plate spline modeling of elastic deformations,” PatternRecognition, Vol. 36(8), pp. 1859-1867, 2003.[7] Z.M. Kovács-Vajna, “A fingerprint verification system basedon triangular matching and dynamic time warping,” IEEETransactions on Pattern Analysis and Machine Intelligence,Vol. 22(11), pp. 1266-1276, 2000.[8] F.L. Bookstein, “Principal warps: Thin plate splines and thedecomposition of deformations,” IEEE Transactions on PatternAnalysis and Machine Intelligence, Vol. 11(6), pp. 567-585.[9] A.K. Jain, L. Hong and R. Bolle, “On-line fingerprint verification,”IEEE Trasactions on Pattern Analysis and MachineIntelligence, Vol. 19(4), pp. 302-314, 1997.[10] I.L. Dryden and K.V. Mardia, “Statistical Shape Analysis,”John Wiley and Sons, 1998.[11] D. Maio, D. Maltoni and R. Cappelli and J. L. Wayman andA. K. Jain, “Fingerprint verification competition,” Proceedingsof the 16th International Conference on Pattern Recognition(ICPR), Québec City, Canada, 2002.[12] C. Watson, P. Grother and D. Cassasent, “Distortion-tolerantfilter for elastic distorted fingerprint matching,” Proceedingsof SPIE Optical Pattern Recognition, pp. 166-174, 2000.[13] A.K. Jain, S.Prabhakar and S.Chen, “Combining MultipleMatchers for a High Security Fingerprint Verification System,”Pattern Recognition Letters, Vol 20, No. 11-13, pp.1371-1379, 1999.