Parabolic Curve fitting as an 3D Trajectory Estimation of the Soccer ...

Parabolic Curve fitting as an 3D Trajectory Estimation of the Soccer BallKyuhyoung Choi and Yongduek SeoSogang University, {Kyu, Yndk}@sogang.ac.krAbstract A soccer ball is projected to an image as a small whitish round blob. With multiplesynchronized views, its 3D position can be estimated. In this paper, the ball trajectory ismodeled as a sequence of 3D parabolic(ballistic) curves from multiple views. With this trajectoryestimation method, lossless tracking is guaranteed for pretty long video clips as shown in ourexperiments.1 IntroductionThe most difficulty in visual tracking of soccer ballcomes from its size. Its small size with fast motionmakes its features useless becoming small whitishand noisy ellipse which makes it hard to keep trackingthe ball without aid of noise removal [1, 2]. Anotherdifficulty is from its interaction with playerswhich makes it generally accepted that ball trackinghas two modes of status, that is, visible and invisible.During visible mode, it shows ballistic motionof elastic sphere while it is occluded by players andhardly separated as independent image blob duringthe other mode. So tracking ball becomes findingthe trajectory for each visible mode. For extractionof 2D ball trajectory, [3] used triplet seeds growingto trajectory candidates. One of the candidatesbest satisfying constant acceleration model is selectedas the trajectory for the segment. However,constant acceleration in 2D does not mean any ofthe ball’s physics though it is the best choice. In oursetting, we use 3D ball trajectory as an 3D paraboliccurve with the acceleration of gravity. Some landmarkson the pitch gives clues to reconstruct 3Dinformation which leads to 3D ball tracking [4]. Ifthe ball can be viewed from the multiple cameras,ideally the 3D rays from each camera centers willmeet at one point which is the center of the ball.However, this can be applied to clutters too. Wehave to exclude less probable 3D points(clutters)one by one until we get the most probable one(true3D ball point). This can be done by evaluating theconsistency of a sequence of 3D points. As applications,3D tracking results can be further processedto lead to enriched broadcast and game analysis [5].The purpose of our trajectory estimation approachis to achieve lossless tracking given a batch of longvideo sequence such as a half soccer match. Therest of this paper is organized as following: Section 2deals with our soccer tracking environment and preimageprocessing. The ball trajectory extraction algorithmis discussed in Section 3. Section 4 providesexperimental results and finally Section 5 concludesthis paper.2 Getting Observation DataThe input data to ball tracking is taken fromsynchronized N(four in our cases as in Figure 1)multiple views of static cameras which means 3Dtracking. Each camera is static covering someFigure 1: Multiple cameras surrounding the pitch.part of the pitch. All the camera images arebackground-subtracted and connected-componentlabeledso that an object is observed as an imageblob(s)(middle and right columns of Figure 2).Using the lines drawn on the pitch, each camera calibrationis done so that a corresponding 3D ray canbe computed when a point in 2D image is given.1

Figure 2: Extracting ball candidate blobs. n throw corresponds to n th camera. Left, middle andright columns show original, foreground and ballcandidates images respectively. Ball positions aremanually marked for readers better understanding.3 Ball Trajectory EstimationBall tracking is also a process of filtering one mostlikely sequence of 3D positions out of a sequence ofnoisy multi-view 2D observation sets. Before goingfurther let us define some sets.Q ∗ 1:T = {q ∗ t } T t=1(1)S = {Q t1 :t 2|1 ≤ t 1 ≤ t 2 ≤ T } (2){ {q }i It TU = t i=1}(3)t=1{ { } NP = {p n t (h)} H nh=1(4)n=1} Tt=1where p and q denote points in 2D and 3D respectively,N is the number of cameras and H is thenumber of observations. Q ∗ 1:T is the optimal trajectory,S is a set of 3D trajectory segments, U isa set of 3D ball candidates and P is a sequence ofnoisy multi-view 2D observation sets. As in reverseorder, U is built from P , then S is from U as wellas Q ∗ 1:T is extracted from S.To build U from P , for each time t, 3D ball candidatesare generated from all the possible pairs ( )C2Nof synchronized N views. Given the camera parameters,a point, for example the one q g on the pitchground, on a ray from the camera center, q c areprojected on a point on the image. Ideally if thereexists a 3D object(as a point) and a pair of cameraproject it, the rays from each camera center tothe projected point meet at the 3D point. However,due to some noise, the rays may not meet each otherposing some distance. If the distance is tolerable,the mid-point between the rays is taken as a 3Dball candidate. The mid-point q mid between tworays passing through two points q1 c and q g 1 , and qc 2and q g 2 respectively is computed as following.where⎡A =⎢⎣q1,2 mid = qclosest1 + q2closest]2(5)= A −1 B (6)[ qclosest1q closest2q1 c (z) 0 µ g,c1 (z) 0 0 00 q1 c (z) µ g,c1 (z) 0 0 00 0 0 q2 c (z) 0 µ g,c2 (x)0 0 0 0 q2 c (z) µ g,c2 (y)µ c,g1 (x) µ c,g1 (y) µ c,g1 (z) µ g,c1 (x) µ g,c1 (y) µ g,c1 (z)µ c,g2 (x) µ c,g2 (y) µ c,g2 (z) µ g,c2 (x) µ g,c2 (y) µ g,c2 (z)(7)B = (q g 1 (x) qc 1 (z) , q g 1 (y) qc 1 (z) , q g 2 (x) qc 2 (z) , q g 2 (y) qc 2 (z) , 0, 0) T(8)q1 closest and q2 closest are the points on the two raysclosest to each other and µ a,bi = qi a − qb i . Then U isthe set of mid-points :{ { ()} } N TU = qm midN (n),m N (n+1) p j m N (n) (t) , pk m N (n+1) (t) n=1t=1(9)where 1 ≤ j ≤ Hm t N (n) , 1 ≤ k ≤ Ht m N (n+1) andm N (n) = MAX (mod (n, N + 1) , 1).From U, S is built by extending all the possibletriplets, three consecutive 3D ball candidates,as long as possible to give trajectory segment candidates.A sequence of three ball candidates in U isqualified to be a triplet if their acceleration and velocitiesshow that of ballistic motion under gravity:S =whereβ t =δ t =ϕ t ={}{q t } t 2t=t 1|β t δ t δ t+1 ϕ t > 0, ∀t : t 1 ≤ t ≤ t 2 − 2(10){ 1 Tlβ< q t+2 (z) − 2q t+1 (z) + q t (z) < Tβ(11)u0 otherwise{1 Tlδ< ‖q t+1 − q t ‖ 2< Tδu (12)0 otherwise{ (1 cos −1 (qt+1−q t)·(q t+2−q t+1)‖q t+1−q t‖ 2 ‖q t+2−q t+1‖< T ϕ2)2 (13)0 otherwiseTo get Q ∗ 1:T from S, the longest one among the segmentcandidates is chosen and fitted to a parabolic⎤⎥⎦

curve parameterized by Θ.the entire length of the second sequence is about 50minutes covering a half of a soccer match which weΘ = {a, b, c, d, e, f, g|q t (x) = at + b, q t (y) = cq (x) + d, aim at.q t (z) = eq (x) 2 + fq (x) + g}Clutters caused from some audience mimic the(14) air ball and produce false positives. However, theyare likely filtered off since their long-term behaviorsThe nearest segments to the both ends of thelongest are merged into the longest if its fitness tothe curve is tolerable, then the curve is updated consideringthe new support. After iterations as shownin Algorithm 1, is estimated a parabolic representativeof the ball motion for a certain period bothends of which correspond to the points where thehardly satisfies the ballistic motion constraints, e.g.,they never come down on the ground.When the ball rolls over the ground, its trajectoryis theoretically an extreme case of a 3D parabola.So the estimation of parabolic parameters is verysensitive to noise so that most such cases are notestimated.3D curve meets the pitch ground. Since the coefficientsof 3D parabolic curve equation is estimatedQ ∗ 1:T is a function of time.5 ConclusionAlgorithm 1 Growing a sequence of 3D points supportinga 3D parabolic curveRequire:{q} S = arg max (j − i){q t } j t=i ∈SEquation 14Ensure:AreOverlapped(a, b) returns T (true) if the sequencesa and b are temporally overlapped.AreConsistent(a, b) returns T if the sequences aand b are qualified to be parts of the same sequence.while isChanged = T and isOutOfRange = FdoisChanged := FisOutOfRange := Ffor i, {q} i ∈ S doif AreOverlapped({q} i , {q} S ) = T andAreConsistent({q} i , {q} S ) = T then{q} S := {q} S ∪ {q} iΘ S := arg max p ({q} S|Θ)ΘisChanged := Tif S = ∅ or q S 1 (z) ≤ 0 and q S |{q} S | (z) ≤ 0thenisOutOfRange := Tend ifbreakend ifend forend while4 ExperimentsExperiments were carried out on two video sequences.One is 500-frame long and of SD size 720× 480 (Figure 3) while the other has 6000 framesand size of HD, 1280 × 720 (Figure 4). ActuallyThis paper proposed a 3D soccer ball trackingmethod from multi-camera environment. A soccerball is projected to an image as a small whitishround blob. With multiple synchronized views,its 3D position can be estimated. In this paper,the ball trajectory is modeled as a sequence of3D parabolic(ballistic) curves from multiple views.With this trajectory estimation method, losslesstracking is guaranteed for pretty long video clipsas shown in our experiments. Currently ourmethod uses a greedy approach which finds thebest trajectory segment first then the second best,and so forth. However, globally optimal approachwith probabilistic frame will be desirable.Acknowledgements This research is accomplishedas the result of the research project forculture contents technology development supportedby KOCCAReferences[1] Yu, X., Xu, C., Leong, H., Tian, Q., Tang, Q.,Wan, K.: Trajectory-based ball detection andtracking with applications to semantic analysisof broadcast soccer video. In: ACM MM03,Berkeley. (2003) 11–20[2] Tong, X.F., Lu, H.Q., Liu, Q.S.: An effectiveand fast soccer ball detection and trackingmethod. In: ICPR (4). (2004) 795–798[3] Yan, F., Kostin, A., Christmas, W., Kittler,J.: A novel data association algorithm for objecttracking in clutter with application to tennisvideo analysis. In: CVPR ’06: Proceedingsof the 2006 IEEE Computer Society Conferenceon Computer Vision and Pattern Recognition,Washington, DC, USA, IEEE Computer Society(2006) 634–641

[4] Kang, J., Cohen, I., Medioni, G.: Soccer playertracking across uncalibrated camera streams.In: Joint IEEE International Workshop on VisualSurveillance and Performance Evaluation ofTracking and Surveillance (VS-PETS). (2003)[5] Yan, X., Yu, X., Hay, T.S.: A 3d reconstructionand enrichment system for broadcast soccervideo. In: MULTIMEDIA ’04: Proceedings ofthe 12th annual ACM international conferenceon Multimedia, ACM Press (2004) 746–747[6] Seo, Y., Choi, S., Kim, H., Hong, K.: Whereare the ball and players? soccer game analysiswith color-based tracking and image mosaick.In: Proc. Int. Conf. on Image Analysis and Processing,Florence, Italy. (1997)(a) 30 th frame(b) 130 th frame(c) 240 th frame(d) 300 th frameFigure 3: Result images of ball tracking. The projectedposition of the estimated ball on each camerais marked as red circle

(a) 200 th frame(b) 850 th frame(c) 1000 th frame(d) 1300 th frame(e) 2300 th frame(f) 5000 th frame(g) 5400 th frame(h) 5800 th frameFigure 4: Another example of ball tracking. Four cameras are located around the pitch almost symmetrically.The four displayed numbers are frame index and X, Y and Z coordinates respectively in order.When the ball position is not estimated the word ”Disappeared” is displayed.