Fast Detection of Multi-View Face and Eye Based on Cascaded ...

MVA2005 IAPR Conference on Machine VIsion Applications, May 16-18, 2005 Tsukuba Science City, Japan3-25<strong>Fast</strong> <strong>Detection</strong> <strong>of</strong> <strong>Multi</strong>-<strong>View</strong> <strong>Face</strong> <strong>and</strong> <strong>Eye</strong><strong>Based</strong> on Cascaded ClassifierJung-Bae Kim, Seok-Cheol Kee <strong>and</strong> Ji-Yeon KimComputing Lab., Samsung Advanced Institute <strong>of</strong> TechnologyP.O.BOX. 111, Suwon, Gyeonggi-do 440-600Korea{jung-bae.kim, sckee, jiyeon.kim}@ samsung.comAbstractIn ourmulti-view face <strong>and</strong> eye detection, we use acascaded classifier trained by gentle AdaBoost algorithm,one <strong>of</strong> the appearance-based pattern learningmethod. Specifically, in orderto detect multi-view face,we propose a special cascaded classifier usingcoarse-to-fine search, simple-to-complex search, <strong>and</strong>parallel-to-separated search. In orderto detect eye, wepropose a four-step eye detection method. Using proposedmethods, we got face <strong>and</strong> eye detection ratios as99.5%, 88.3%, respectively for7different DBs including17,018various multi-view faces.1 Introduction<strong>Multi</strong>-view face <strong>and</strong> eye detection technology isacore technologyto face recognition <strong>and</strong> image retrieval.There can be three kinds<strong>of</strong>head rotation. X-axisrotationisup-down nodding, y-axisrotation isout-<strong>of</strong>-planerotation (pr<strong>of</strong>ile view), <strong>and</strong> z-axisrotation isin-planerotation (left-righthead leaning).Rowley et al. detected z-axis rotated faces [1].Schneiderman etal. detected y-axisrotated [2]. In orderto detecty-axisrotated faces<strong>and</strong> z-axisrotated facesatthe same time, Li etal. rotated the inputimage withthree differentangleswith respectto z-axis<strong>and</strong> applieddetector-pyramid detecting y-axis rotated faces [3].Jones<strong>and</strong> Viola made y-axisrotated face detector<strong>and</strong>z-axisrotated face detectorindependently[4]. W u etal.used a pose estimatorin earlystage <strong>and</strong> applied narrowview face detectorsaccording to the result<strong>of</strong>the poseestimator [5]. [1] <strong>and</strong> [4] also used pose estimatormethod. However, errorsfrom pose estimatordeterioratedthe face detection rate. So far, variousmethodsonmulti-view face detection are proposed. However, theyshow onlylimited performance.To solve these problems, we use basicallya classifiertrained by gentle AdaBoostalgorithm, one <strong>of</strong>the appearance-basedpa tern learning method. In order todetectmulti-view face, we propose a special cascadedclassifier structure using coarse-to-fine search, simple-to-complexsearch<strong>and</strong> parallel-to-separated search.On the otherh<strong>and</strong>, itisa very challenging work todetecteye pairsin variousreal environments. There maybe many varying factorssuch asscale, pose, rotation,closed eye, illumination, glassreflection, occlusion, etc.Recentresearchesproposed variousmethodsto detecteye pairs. However, most<strong>of</strong>all consideronlysome part<strong>of</strong>variation. Kawaguchi etal. <strong>and</strong> Baskan etal. usedface structure knowledge such as Hough transform,symmetry detector, projection analysis [6][7]. Thesemethodsdo notconsiderphysical featuresthateyeshavemuch variation. In addition, these methodsuse a binarizedimage with threshold value. <strong>Eye</strong> c<strong>and</strong>idate setresulted from thresholding hasreal eye with low probability.Asourexperiment, itisonly 90.8%. Itmeansthatany classifierfollowing the binarization processcannothave eye detection ratio overthan 90.3%. Luceyetal. tried to use a learning method on eye detection [8].However, itcould notdiscriminate between a thick glassframe <strong>and</strong> a closed eye since itused onlyeye information.So far, variouseye detection methodsare proposed.However, theyalso show limited performance.In thispaper, we propose a four-step eye detectionmethod. In firststep, we restrictthe both eyes’c<strong>and</strong>idateareasfrom an inputimage. In second step, we detectboth eyes’c<strong>and</strong>idatesusing an eye classifier. In thirdstep, we extracteye pairc<strong>and</strong>idates’feature using an eyepairclassifier. In fourth step, we decide an eye pairbased on the three featuresextracted from previoustwosteps.2 Gentle AdaBoostWe adopt an AdaBoost algorithm to implementmulti-view face detection <strong>and</strong> eye detection. Thisalgorithmwasfirstproposed byFreund <strong>and</strong> Schapire [9]<strong>and</strong>itbecame popularin the face detection field afterViola<strong>and</strong> Jones[10]. AdaBoostisa very e fective learningalgorithm thatorganizessimple <strong>and</strong> fastweak classifiersasa weighted sum <strong>and</strong> rendersa faststrong classifierhaving a high successrate. Specifically, we use a gentleAdaBoost[9], a gentle version <strong>of</strong>real AdaBoost, asfollows:1) Given N examples (x 1 , y 1 ), … , (x N , y N ) withkx R , yi{1,1}2) Startwith weightsw i = 1/N, i = 1, … , N.3) Repeatfork = 1, … , K.(a) Find a weak classifierf k (x), which minimizes Ni1w ( y fiik( x ))(b) Set w w exp(y f ( x )), i 1, Nii ii k i,renormalize weightsso that 12N w ii1<strong>and</strong>116

4) Output the classifier Ksign f k( x) k 1A weak classifier consists <strong>of</strong> a threshold value <strong>and</strong> asimple feature, as shown in Fig. 1. The specific valuesare determined according to an iterative learning. Thesimple features are designed to detect an edge or a line<strong>of</strong> the face easily. Several supplementary features areadded to Viola <strong>and</strong> Jones’ features [10].In the learning stage, all possible positions, sizes, <strong>and</strong>types <strong>of</strong> features are considered within a 2424 window.There are a total number <strong>of</strong> 117,400 features with somerestrictions to their freedom such as minimum area. Aface/non-face strong classifier is applied to the inputimage in all possible positions with all possible sizes inorder to detect all the faces with various positions <strong>and</strong>sizes.However, accordingaswerotatethesimplefeature<strong>of</strong>adetectorwith90°orperform amirroringoperationonthesimplefeature, wegetotherdetectors.Forexample,aleft-view detectorcanbearight-view detector.Also,12 frontal-view detectorscanbemadefrom only2 detectors.Astheresult,thenumber<strong>of</strong>facedetectorstobetrainedisshowninTableI.3.2 The Structure <strong>of</strong> <strong>Multi</strong>-<strong>View</strong> <strong>Face</strong> DetectorInordertodetectmulti-view face, whenweuse12 or72 facedetectorsindependently, weneed12 timesor72times<strong>of</strong>computationcomparingtoasinglefacedetector.Ifweadoptaposeestimator, therehappensaposeestimator’errorproblemaswementioned.Weproposeanew multi-view detectorbased on a specialcascadedclassifier structure using coarse-to-fine search, simple-to-complexsearch, <strong>and</strong> paralel-to-separated searchasshowninFig.2.(a)(b)Fig. 1. Haar-like simple features: (a) edge features;(b)line features.3 <strong>Multi</strong>-<strong>View</strong> <strong>Face</strong> <strong>Detection</strong>3.1 Detectable Rotation AngleFor x-axis rotation, we need only 2 detectors. Theyare a down-view face detector covering [-60, -20]<strong>and</strong>afrontal/upwardfacedetectorcovering[-20, 50].Fory-axisrotation, weneed3 detectorsincludingaleft-viewfacedetector, afrontalfacedetector<strong>and</strong> aright-viewface detector.Theircovering anglesare [-90, -20],[-20, 20], [20, 90], respectively.Forz-axisrotation, wedealwithalrotationcovering[-180, 180].However, duringst<strong>and</strong>ing, apersoncanlean his/her head with [-45, 45].W e cal“Basicmode”<strong>of</strong>z-axisrotationas[-45, 45]<strong>and</strong>“Extendedmode”<strong>of</strong>z-axisrotationas[-180, 180].W hen wedesign adetectorcovering 30°on z-axisrotation, 12 detectorscoverstheextendedmode, [-180°,180°].Forthe basic mode, 3 detectorsare sufficient.Thisiscaledas“MethodI.”W henwedesignadetectorcovers45°, 8detectorscancoverstheextendedmode.Forthebasicmode, 2 detectorsaresuficient.Thisiscaled as“Method II.”W hen weconsiderx, y, z-axisrotation, thenumber<strong>of</strong>facedetectorsneededisshowninTableI.TableI.Thenumbers<strong>of</strong>individualfacedetectorsneededinmulti-view facedetection.MethodIMethodIINumber<strong>of</strong>faceDetectorsneeded.Number<strong>of</strong>facedetectorstobetrainedBasicmode 18=233 10=25Extendedmode72=2312 10=25Basicmode 12=232 6=23Extendedmode48=238 6=23(a) (b) (c)Fig.2.Threemethodsrenderingacascadedclassifierusedin the multi-view face detector;(a)Coarse-to-fine search; (b) Simple-to-complexsearch;(c)Paralel-to-separatedsearch.Coarse-to-finesearchisthatawhole-view classifierislocatedinearlystage<strong>and</strong>narrower-view classifiersarelocated in latestage.Simple-to-complex search isthateasierclassifiersare located in early stage <strong>and</strong> morecomplexclassifiersarelocatedinlatestage.Usingthesetwostages, wecanspeedamulti-view detectorupsincemostnon-facesareeliminatedinearlystages.Paralel-to-separated search isthataldetectorsarearranged in paraleluntilK stage<strong>and</strong> each detectorisarrangedseparatelyfrom K+1stage.Usualy, whenaninputisentered<strong>and</strong>afaceisdetectedsuccessfulyinacertainstage<strong>of</strong>aview, theinputmovestonextstage<strong>of</strong>thesameview tobetested.Paralelarrangementmeansthatwhenafaceisnotdetected, theinputmovestothesamestage<strong>of</strong>nextview.Separatedarrangementmeansthatwhenafaceisnotdetected, nomoreprocedureisneeded.Wejustdecidetheinputisanon-face.Usingthismethod, aninputimageisdecidedasacertainviewinearlystage, <strong>and</strong>thenweconcentrateonwhethertheinputisafaceornotinlatestage.InFig.2, s st<strong>and</strong>sforsuccessinfacedetection<strong>and</strong>f st<strong>and</strong>sforfailure.We combine three cascading methods into amulti-view facedetectorasshowninFig.3.In1~2 stage,whole-view faceisdetected.In3~4stage, facegroupssuchasuprightface, left-leanedface, right-leanedfacearedetected.In5~M stage, alview facearedetected.Also, paralelsearchesare implemented untilK stage<strong>and</strong>separatedsearchesareimplementedfrom K+1stage.InFig.3, NF st<strong>and</strong>sfornon-face, s st<strong>and</strong>sforsuccessinfacedetection, <strong>and</strong>f st<strong>and</strong>sforfailure.117