A novel fuzzy clustering algorithm based on a fuzzy scatter matrix ...

More documents

Recommendations

Info

646 K.-L. Wu et al. / Pattern Recognition Letters 26 (2005) 639–652 ðx j xÞðx j xÞ t If "i, a i ¼ x, C b X ¼ P n j¼1 , and "j, nkx j xk 2 Þ kx j xk > 0, then we get the following equation u t o 2 J a¼ða1 a u oa i oa a k ;a 2 ;...;a cÞ where a i¼x;8i P ¼ 4m c X n i¼1 u t 2 a i ðx j xÞ cðm 1Þ kx j xk 2 þ 2n c m X c i¼1 j¼1 ð1 g i Þu t 2m C X a i I ss u ðm 1Þ ð1 g i Þ ai : ð20Þ For simplifying the analysis, we assume that "i, g i = g and "j, "k, q(x j ,a k ) > 0. In this way, we can ignore the update equations for g i to reduce the complexity of the analysis on FCS. Thus, Eq. (19) turns into (21) as follows: A good <strong>clustering</strong> method should have the robust ability to tolerate noise and outliers. In this section, we use the gross error sensitivity and influence function (Huber, 1981) to show that our weighted cluster center update equation is robust to noise and outliers. Let {x 1 ,...,x n } be an observed data set of real numbers and h is an unknown parameter to be estimated. An M-estimator (Huber, 1981) is generated by minimizing the form X n j¼1 qðx j ; hÞ; ð22Þ where q is an arbitrary function that can measure the loss of x j and h. Here, we are interested in a location estimate that minimizes X n j¼1 qðx j hÞ ð23Þ u t a o 2 J oa i oa k u a ¼ ! 4m 2 X n X ðS j Þ m 1 c l m ij m 1 ut a i ½ð1 gÞa i ðx j gxÞŠ j¼1 i¼1 þ 2ð1 gÞ Xc i¼1 X n j¼1 l m ij ut a i I 2m X n m 1 j¼1 l m ij ½ða ! i x j Þ gða i xÞŠ½ða i x j Þ gða i xÞŠ t ð1 gÞqðx j ; a i Þ P n u j¼1 lm ai : ij ð21Þ From Eq. (21), we know that if g approaches negative infinity, then any FCS solution will be stable. This is an unacceptable result. Similarly, if g approaches positive infinity, any FCS solution will be unstable. This is also unacceptable. Therefore, we can roughly set the range for g with 1
K.-L. Wu et al. / Pattern Recognition Letters 26 (2005) 639–652 647 uðx hÞ ICðx; F ; hÞ ¼R u0 ðx hÞdF X ðxÞ ; ð25Þ where F X (x) denotes the distribution function of X. If the influence function of an estimator is unbounded, a noise or outliers might cause trouble. Identically, if the u function of an estimator is unbounded, noise and outliers will cause trouble. Many important robustness measures can be observed from the influence function. One of the important measures is the gross error sensitivity c , defined by c ¼ sup jICðx; F ; hÞj: ð26Þ x This quantity can interpret the worst approximate influence that the addition of an infinitesimal point mass can have on the value of the associated estimator. Let the loss between the data point x j and ith cluster center a i be qðx j a i Þ¼l m ij kx j a i k 2 g i l m ij ka i xk 2 ð27Þ and uðx j a i Þ¼ o qðx j a i Þ oa i ¼ 2l m ij ðx j a i Þ 2g i l m ij ða i xÞ: ð28Þ By solving the equation P n j¼1 uðx j a i Þ¼0, we have the result shown in Eq. (14). Thus, the FCS cluster center is an M-estimator with the loss function (27) and u function (28). Note that, the u function of our estimator is a function of l m ij which depends on the fuzzifier m. We will show that the FCS cluster center is robust to noise and outliers when m is large. For a given data set, we will produce the FCS cluster centers {a 1 ,...,a c }, the parameters {g 1 ,...,g c } and the sample mean x by processing the FCS <strong>clustering</strong> <strong>algorithm</strong>. The relative influence (the u function) of an individual observation x toward the ith cluster center can be defined as uðx a i Þ¼ 2ðl i ðxÞÞ m ½ðx a i Þ g i ða i xÞŠ; ð29Þ where l i ðxÞ ¼ ðkx a i k 2 g i ka i xk 2 m Þ 1 1 P c k¼1 ðkx a : ð30Þ kk 2 g k ka k xk 2 m Þ 1 1 Note that, l i (x)=1ifkx a i k 2 6 g i ka i xk 2 . Suppose that x 2 R. For an extremely large or small x, we will have kx a i k 2 P g i ka i xk 2 and hence l i (x) 2 (0, 1). In general, the extremely large or small x will fall outside any one of c cluster kernels and have <strong>fuzzy</strong> membership value l i (x) 2 (0, 1) or more precisely, l i (x) will be very closed to 1/c and hence l i (x) m will be very closed to zero when m is large. Although ðx a i Þ g i ða i xÞ is a monotone increasing function of x, ðl i ðxÞÞ m ½ðx a i Þ g i ða i xÞŠ will be very closed to zero for an extremely large or small x when m is large. Thus, the u function of the FCS cluster center will be bounded for an extremely large or small x when m tends to infinity. Therefore, for a large m case, only the data point x inside the ith cluster kernel with l i (x) = 1 will have an influence on the ith FCS cluster center a i and the data point x falls on the crisp boundary of the ith cluster center will have the gross error sensitivity c < 1. The above discussions give a theoretical foundation to asseverate the FCS <strong>algorithm</strong> to be robust to noise and outlier when m is large. The following is a simple example. For the data set shown in Fig. 2(a), we implement FCS with b = 0.1 and m = 2 to produce the set of {a 1 ,a 2 ,a 3 }, {g 1 ,g 2 ,g 3 }andx. For a given data point x, the membership function l 1 (x) (Eq. (30)) is illustrated in Fig. 3(a) and the u function uðx a 1 Þ¼ 2ðl 1 ðxÞÞ 2 ½ðx a 1 Þ g 1 ða 1 xÞŠ is illustrated in Fig. 3(d). When m = 2, an extremely large or small x will have a large influence on a 1 as shown in Fig. 3(d). Equivalently, we implement FCS with m = 3 and m = 4 to have other sets of {a 1 ,a 2 ,a 3 } and {g 1 ,g 2 ,g 3 } to illustrate the l 1 (x) inFig. 3(b) and (c), respectively. The corresponding u functions are illustrated in Fig. 3(e) and (f). Fig. 3 shows that the influence of an extremely large or small x will become small when m increases. These numerical results coincide to our theoretical analysis. 6. Numerical examples We implement the Normal-4 data set which was proposed by Pal and Bezdek (1995). Normal-4 is a four-dimensional data set with the sample size
Page 1 and 2: Pattern Recognition Letters 26 (200
Page 3 and 4: K.-L. Wu et al. / Pattern Recogniti
Page 7: K.-L. Wu et al. / Pattern Recogniti

A novel fuzzy clustering algorithm based on a fuzzy scatter matrix ...

Create successful ePaper yourself

Delete template?

Save as template?