A novel fuzzy clustering algorithm based on a fuzzy scatter matrix ...

640 K.-L. Wu et al. / Pattern Recognition Letters 26 (2005) 639–652
The <strong>clustering</strong> applications in various areas have
been well documented (Duda and Hart, 1973; Jain
and Dubes, 1988; Kaufman and Rousseeuw,
1990). In these <strong>clustering</strong> methods, the hard
c-means (or k-means) and <strong>fuzzy</strong> c-means (FCM)
<strong>clustering</strong> <strong>algorithm</strong>s are the most well-known
methods (Bezdek, 1981; Jain and Dubes, 1988;
Yang, 1993). Most of these methods are <strong>based</strong>
on minimizing the within-cluster scatter matrix
trace. The within-cluster scatter matrix trace can
be interpreted as a compactness measure with a
within-cluster variation. Because the <strong>clustering</strong> results
obtained using k-means and FCMare
roughly spherical with similar volumes, many <strong>clustering</strong>
<strong>algorithm</strong>s such as the Gustafson–Kessel
(G–K) <strong>algorithm</strong> (Gustafson and Kessel, 1979),
the sum of all normalized determinants (SAND)
<strong>algorithm</strong> (Rouseeuw et al., 1996), the minimum
scatter volume (MSV) and minimum cluster volume
(MCV) <strong>algorithm</strong>s (Krishnapuram and Kim,
2000), the unsupervised <strong>fuzzy</strong> partition-optimal
number of classes (UFP-ONC) <strong>algorithm</strong> (Gath
and Geva, 1989), etc. were proposed to accommodate
elliptical clusters with different volumes.
These <strong>algorithm</strong>s are all <strong>based</strong> on a within-cluster
scatter matrix with a compactness measure.
The concept of adopting a separation measure
in <strong>clustering</strong> is used widely in solving cluster validity
problems such as the separation coefficient proposed
by Gunderson (1978), the XB index
proposed by Xie and Beni (1991), the FS index proposed
by Fukuyama and Sugeno (1989), the SC
index proposed by Zahid et al. (1999), F HV (<strong>fuzzy</strong>
hyper-volume) and P D (partition density) indexes
proposed by Gath and Geva (1989), etc. Özdemir
and Akarun (2001) proposed an inter-cluster separation
(ICS) <strong>clustering</strong> <strong>algorithm</strong> that involves a
separation measure in the ICS objective function.
Because the between-cluster scatter matrix trace
can be interpreted as a separation measure with a
between-cluster variation, maximization of the between-cluster
scatter matrix trace will induce a result
with well-separated clusters (Yang et al., 2003).
In this paper, we propose a <strong>novel</strong> <strong>fuzzy</strong> <strong>clustering</strong>
<strong>algorithm</strong>, called the <strong>fuzzy</strong> compactness and
separation (FCS) <strong>algorithm</strong>. The FCS objective
function is <strong>based</strong> on a <strong>fuzzy</strong> scatter matrix. The
FCS <strong>algorithm</strong> is derived by minimizing the compactness
measure and simultaneously maximizing
the separation measure (Yang et al., 2003). The
compactness is measured using a <strong>fuzzy</strong> within-cluster
scatter matrix. The separation is measured
using a <strong>fuzzy</strong> between-cluster scatter matrix trace.
In k-means, data points always have crisp membership
values of zero or one. Although, FCMallows
the data points to have <strong>fuzzy</strong> membership values
between zero and one, it does not exactly produce
a zero or one for the membership values. In the
proposed FCS <strong>algorithm</strong>, crisp and <strong>fuzzy</strong> membership
values could co-exist. These FCS properties
will be discussed. We will also show that, when the
weighting exponent m is large, the FCS <strong>algorithm</strong>
is more robust to noise and outliers than FCM.
The theoretical analysis on FCS will be investigated.
Yu et al. (2004) gave a theoretical upper
bound for the weighting exponent m in FCMin
which the grand sample mean x is a unique optimizer
of the FCMobjective function. In this paper,
we will show that FCS with the different cluster
kernel characteristic can avoid the situation in
which x is a unique optimizer of the FCS objective
function. We also studied the optimality tests.
These results will be used as the parameter selection
for FCS. The paper is organized as follows.
In Section 2, the (crisp) scatter matrix is extended
to the <strong>fuzzy</strong> scatter matrix. Some <strong>clustering</strong> <strong>algorithm</strong>s
<strong>based</strong> on the within-scatter matrix are then
reviewed. In Section 3, we propose the <strong>novel</strong> <strong>fuzzy</strong>
<strong>clustering</strong> <strong>algorithm</strong> <strong>based</strong> on the <strong>fuzzy</strong> withinand
between-scatter matrix. Section 4 gives our
theoretical analysis on the optimality tests and
the FCS parameter selection. Section 5 gives the
robust properties of FCS <strong>based</strong> on the gross error
sensitivity and influence function. Some numerical
examples are presented in Section 6. Conclusions
are made in Section 7.
2. Clustering <strong>algorithm</strong>s <strong>based</strong> on a within-cluster
scatter matrix
Let X ={x 1 ,...,x n } be a data set in an s-dimensional
Euclidean space R s and let c be a positive
integer larger than one. A partition of X into c
clusters can be presented using mutually disjoint
sets X 1 ,...,X c such that X 1 [[X c = X or