A novel fuzzy clustering algorithm based on a fuzzy scatter matrix ...

More documents

Recommendations

Info

642 K.-L. Wu et al. / Pattern Recognition Letters 26 (2005) 639–652 According to Eq. (5), we have P n j¼1 lm ij a i ¼ P n j¼1 lm ij x j. Thus, we can create the property that S FT = S FW + S FB where a i is defined by Eq. (5). This property is exactly the same as S T = S W + S B for the (crisp) scatter matrix. Fuzzy <strong>clustering</strong>s including FCM(Bezdek, 1981), alternative FCM(Wu and Yang, 2002; Yang et al., 2002), G–K (Gustafson and Kessel, 1979), SAND (Rouseeuw et al., 1996), MCV (Krishnapuram and Kim, 2000), and UFP-ONC (Gath and Geva, 1989), etc., are all <strong>based</strong> on the <strong>fuzzy</strong> within-cluster scatter matrix S FW . It is known that the FCM<strong>clustering</strong> <strong>algorithm</strong> is created by minimizing the objective function J FCM ¼ trðS FW Þ¼ Xc i¼1 X n j¼1 with the membership update equation l ij ¼ ðkx j a i k 2 Þ 1 m 1 P c k¼1 ðkx j a k k 2 Þ 1 m 1 l m ij kx j a i k 2 ; ð9Þ ð10Þ and the cluster center update Eq. (5). Although tr(S FT ) = tr(S FW ) + tr(S FB ) for a given data set X, tr(S FT ) is not a fixed constant but depends on l ij . Thus, to minimize tr(S FW ), it is not necessary to maximize tr(S FB ). The trace tr(S FB ) of a <strong>fuzzy</strong> set between the cluster scatter matrix can be interpreted as a separation with a cluster variation in between. A maximum value of tr(S FB ) will induce a <strong>clustering</strong> result with separated (distinguishable) clusters. In the next section, an <strong>algorithm</strong> that considers tr(S FW ) and tr(S FB ) simultaneously is introduced. 3. A proposed <strong>fuzzy</strong> <strong>clustering</strong> <strong>algorithm</strong> <strong>based</strong> on tr(S FW ) and tr(S FB ) Fukuyama and Sugeno (1989), Sugeno et al. (1993) had used tr(S FW ) and tr(S FB ) to create an index FS(c) in the cluster validity problem where Pal and Bezdek (1995) gave more discussions in cluster validity for FCM. The validity index FS(c) was formed with FSðcÞ ¼trðS FW Þ ¼ Xc i¼1 X c i¼1 X n j¼1 X n trðS FB Þ l m ij kx j a i k 2 j¼1 l m ij ka i xk 2 : ð11Þ A small FS(c) value will induce a good <strong>fuzzy</strong> <strong>clustering</strong> result with a small <strong>fuzzy</strong> within-cluster variation tr(S FW ) and a large <strong>fuzzy</strong> between-cluster variation tr(S FB ). This will help us find a good cluster number estimate. It is reasonable to have a <strong>clustering</strong> objective function containing a measure of within- and between-cluster variations such as the FS(c) index. However, there does not exist an update equation for a i by differentiating FS(c) with respect to a i . Thus, FS(c) cannot be a <strong>clustering</strong> objective function. In this section, we propose a <strong>novel</strong> <strong>fuzzy</strong> <strong>clustering</strong> objective function which is a modification of the FS(c) index. This could be also a generalization of the FCMobjective function by combining <strong>fuzzy</strong> within- and between-cluster variations. Our goal is to minimize a <strong>fuzzy</strong> within-cluster variation tr(S FW ) and also simultaneously maximize a <strong>fuzzy</strong> between-cluster variation tr(S FB ). We call this a <strong>fuzzy</strong> compactness and separation (FCS) <strong>algorithm</strong>, because the compactness is measured using a <strong>fuzzy</strong> within variation and the separation is measured using a <strong>fuzzy</strong> between variation. Thus, the FCS objective function J FCS is defined as J FCS ¼ Xc X n l m ij kx j a i k 2 i¼1 j¼1 X c X n i¼1 j¼1 g i l m ij ka i xk 2 ; ð12Þ where g i P 0. Note that, J FCS = J FCM when g i =0 and J FCS = FS(c) when g i = 1. By minimizing J FCS we have the following update equations: ðkx j a i k 2 g l ij ¼ i ka i xk 2 m Þ 1 1 P c k¼1 ðkx ð13Þ j a k k 2 g k ka k xk 2 m Þ 1 1 and P n j¼1 a i ¼ lm ij x j P n j¼1 lm ij P n g i j¼1 lm ij P x g n ; ð14Þ i j¼1 lm ij
K.-L. Wu et al. / Pattern Recognition Letters 26 (2005) 639–652 643 where the parameter g i could be set up with g i ¼ ðb=4Þmin i 0 6¼ika i a i 0k 2 ; 0 6 b 6 1:0: ð15Þ max k ka k xk 2 In <strong>fuzzy</strong> <strong>clustering</strong>, we restrict l ij 2 [0,1]. Because l ij in Eq. (13) might be negative for some data point x j , we make some restrictions on it. For a given data point x j ,ifkx j a i k 2 6 g i ka i xk 2 , then l ij = 1, and l i 0 j ¼ 0, for all i 0 5 i. That is, if the distance between the data points and the ith cluster center are smaller than g i ka i xk 2 , these data points will then belong exactly to the ith cluster with membership value of one. Each cluster in FCS will have a crisp boundary such that all data points inside this boundary will have a crisp membership value l ij 2 {0,1} and other data points outside this boundary will have <strong>fuzzy</strong> membership values l ij 2 [0, 1]. Each crisp boundary will form a hyperball for the corresponding cluster and can be seen as a cluster kernel. Fig. 1 shows a two-cluster data set in which each cluster contains a cluster center and a cluster kernel. The volume of each cluster kernel is decided by the term g i ka i xk 2 . Data points outside the kernel will have <strong>fuzzy</strong> memberships. Note that Özdemir and Akarun (2002) proposed the partition index maximization (PIM) <strong>algorithm</strong> that uses a fixed volume of cluster kernels for each cluster. In our FCS, the volume of the kernel for each cluster is different. This FCS characteristic could catch more information from data according to different volumes with different shapes. In the k-means <strong>algorithm</strong>, each data point has a crisp membership value with l ij 2 {0,1}. Although cluster center <strong>fuzzy</strong> boundary cluster kernel Fig. 1. Clusters obtained by FCS. crisp boundary FCMallows data points to have membership values l ij in the interval [0,1], it has fewer crisp membership values (i.e. zero or one) in FCM, even when the data points are very close to any one of these c cluster centers. The memberships seem to be too <strong>fuzzy</strong> in FCM. In our FCS, crisp and <strong>fuzzy</strong> membership values co-exist. Data points that fall inside any one of c-cluster kernels (i.e. close to any one of c cluster centers) will have crisp memberships and those outside the cluster kernels (i.e. far away from all cluster centers) will have <strong>fuzzy</strong> membership values. To guarantee that no two of these c cluster kernels will overlap, g i is chosen as Eq. (15) such that the parameter b will control the size of each kernel. Since, for all i, 0 1 g i ka i xk 2 ¼ðb=4Þðminka i i 0 6¼i 6 b min i 0 6¼i a i 0k 2 ka i xk 2 Þ@ A maxka k xk 2 k ka i a i 0k ! 2 2 ka i a i 0k 2 6 min i 0 6¼i 2 for all 0 6 b 6 1; we have, for a given data point x j with l ij =1, 2 kx j a i k 2 6 g i ka i xk 2 ka i a i 0k 6 min : i 0 6¼i 2 Note that the value of ka i a i 0k=2 is a half the distance between cluster centers a i and a i 0. Thus, Eq. (15) should guarantee that no two of these c cluster kernels will overlap. If b = 1.0, the FCS <strong>algorithm</strong> will cluster the data set using the largest kernel for each cluster. If b = 0 (i.e. g i = 0), the FCS <strong>algorithm</strong> will cluster the data set with no cluster kernel. This will be equivalent to the FCM<strong>clustering</strong> <strong>algorithm</strong>. The cluster center update Eq. (14) in FCS can be interpreted as a weighted mean of the cluster center update equation of FCMand the grand mean x. We can rewrite Eq. (14) as P n j¼1 a i ¼ lm ij x j= P n j¼1 lm ij g i x : ð16Þ 1 g i The weights of the FCMcluster center and the grand mean x are 1 and g i , respectively. To maximize tr(S FB ), the cluster centers obtained by FCS
Page 1 and 2: Pattern Recognition Letters 26 (200
Page 3: K.-L. Wu et al. / Pattern Recogniti
Page 7 and 8: K.-L. Wu et al. / Pattern Recogniti

A novel fuzzy clustering algorithm based on a fuzzy scatter matrix ...

Create successful ePaper yourself

Delete template?

Save as template?