TESI DOCTORAL - La Salle

Chapter 1. Framework of the thesis
not always available, as some domains are more prone to be organized hierarchically than
others. Possibly due to this fact, not much research has been done on external hierarchical
clustering evaluation (Patrikainen and Meila, 2005). Some examples of the few existing
hierarchical clustering comparison methods are simple layer-wise comparison (Fowlkes and
Mallows, 1983) and cophenetic matrices (Theodoridis and Koutroumbas, 1999), although
they also have their shortcomings (Patrikainen and Meila, 2005). For this reason, the most
extended strategy is to compare the clusterings found at a certain level of the dendrogram
with a partitional ground truth. Unfortunately, this approach does not take in account the
cluster hierarchy in any way, which is clearly not the point if the hierarchical clustering
solution is to be validated as a whole.
Allowing for all these considerations, and provided that the outputs of soft and hierarchical
clustering algorithms can always be converted to hard and partitional clustering
solutions, respectively (see section 1.2.1), the most common cluster validation procedure
consists in comparing hard partitional clustering solutions (i.e. label vectors) with the
same type of ground truths (i.e. comparing cluster labels with class labels) (Strehl, 2002).
The following paragraphs are devoted to a description of some relevant external validity
indices for evaluating hard partitional clustering solutions.
The multiple possible ways for comparing the class labels contained in the ground truth
label vector γ and the cluster labels in a label vector λ can be categorized into two groups
depending on whether they are based on i) object pairwise matching, or ii) cluster matching.
Object pairwise matching cluster validity indices are based on counting how many object
pairs (xi, xj) , ∀ i = j, are clustered together and separately in both γ and λ (the more coincidences,
the higher the similarity between the clustering solution and the ground truth).
Following this rationale, several validity indices have been proposed, such as the Rand index
(Rand, 1971), the Adjusted Rand index (Hubert and Arabie, 1985), the Fowlkes-Mallows
index (Fowlkes and Mallows, 1983) or the Jaccard index, among others—see (Halkidi, Batistakis,
and Vazirgiannis, 2002a; Denoeud and Guénoche, 2006).
Cluster matching cluster validity indices measure the degree of agreement between the
assignment of objects to classes (according to γ) and clusters (as designated by λ). Typical
examples of this kind of validity indices are the Larsen index (Larsen and Aone, 1999), the
Van Dongen index (van Dongen, 2000), variation of information (Meila, 2003), entropy or
mutual information (Cover and Thomas, 1991), to name a few.
In all the experimental sections of this work, the cluster validity index used for evaluating
clustering results is normalized mutual information, denoted as φ (NMI) . This choice is
motivated by the fact that φ (NMI) is theoretically well-founded, unbiased, symmetric with
respect to λ and γ and normalized in the [0, 1] interval —the higher the value of φ (NMI) ,the
more similar λ and γ are (Strehl, 2002). Mathematically, normalized mutual information
is defined as follows:
φ (NMI) k k h=1 l=1
(γ, λ) =
nh,l log
k h=1 n(γ)
)
n(γ k
h
h log n
n·nh,l
(γ )
n h n(λ)
l
l=1 n(λ)
l
log n(λ)
l
n
(1.8)
where n (γ)
h is the number of objects in cluster h according to γ, n (λ)
l is the number of objects
17