TESI DOCTORAL - La Salle

1.2. Clustering in knowledge discovery and data mining
0.6
0.4
0.2
0
1
2
3
−0.2
−0.2 0 0.2 0.4 0.6
(a) Scatterplot of the data of this
toy two-dimensional data set.
4
7
5
9
6
8
Euclidean distance
0.35
0.3
0.25
0.2
0.15
0.1
7 9 8 4 5 6 1 2 3
object index
(b) Dendrogram obtained by the
single-link hierarchical algorithm.
Figure 1.4: A hierarchical clustering toy example: (a) Scatterplot of an artificially generated
two-dimensional data set containing n = 9 objects, each one of them is identified by a
numerical label. (b) Dendrogram resulting of applying the single-link hierarchical agglomerative
clustering algorithm on this data, using the Euclidean distance as the similarity
measure. The dashed horizontal line performs a cut on the dendrogram, yielding a 4-way
partition with an Euclidean distance between clusters ranging between 0.112 and 0.255.
performing a cut through the dendrogram at the desired level of cluster similarity or
by setting the desired number of clusters k, as shown by the dashed horizontal line in
figure 1.4(b).
On the other hand, the overlap of the clusters into which objects are grouped is an
additional factor which allows splitting clustering algorithms into two large categories: hard
and soft algorithms.
1. Hard (aka crisp) clustering algorithms: this type of algorithms partition the data
set into k disjoint clusters, i.e. each object is assigned to one and only one cluster.
In mathematical terms, the result of a hard clustering process on the data set X
is a n-dimensional integer-valued row label vector λ = [λ1 λ2 ... λn], where λi ∈
{1, 2,...,k}, ∀i ∈ [1,n]. That is, the ith component of the label vector (or labeling
for short) contains the label of the cluster the ith object in the data set is assigned
to. For instance, the label vector obtained after applying a classic hard clustering
algorithm such as k-means on the artificial toy data set depicted on figure 1.4(a),
setting k =3,isλ =[222111333]. Notice the symbolic nature of the cluster labels,
as the same clustering result would be represented by label permuted label vectors
such as λ =[111222333] or λ =[333111222].
2. Soft (aka fuzzy) clustering algorithms: they generate a set of k overlapping clusters,
i.e. each object is associated to each of the k clusters to a certain degree. Hence, the
result of conducting a soft clustering process on the data set X is a k × n real-valued
clustering matrix Λ, whose(i,j)th entry indicates the degree of association between
the ith cluster and the jth object. This degree of association is typically expressed
in terms of the probability of membership of each object to each cluster, as done by
10