TESI DOCTORAL - La Salle

1.2. Clustering in knowledge discovery and data mining
indices.
– a predefined and allegedly correct cluster structure, measuring its degree of resemblance
to the obtained clustering solution by means of external cluster validity indices.
– a clustering solution resulting from another clustering process (e.g. a distinct execution
of the same clustering algorithm but using different parameters), measuring
their relative merit so as to decide which one may best reveal the characteristics of
the objects using relative cluster validity indices.
All these three types of evaluation criteria can be used for validating individual clusters,
as well as the output of partitional and hierarchical clustering algorithms (Jain and Dubes,
1988; Halkidi, Batistakis, and Vazirgiannis, 2002a; Halkidi, Batistakis, and Vazirgiannis,
2002b).
As regards their applicability, only internal and relative evaluation criteria are applicable
to evaluate clustering solutions in real-life scenarios. This is due to the unsupervised
nature of the clustering problem, as the class membership of objects is unknown in practice.
However, in a research context –where ‘correct’ category labels presumably assigned by an
expert to the objects in the data set are usually known, but not available to the clustering
algorithm–, it is sometimes more appropriate to use external validity indices, since clusters
are ultimately evaluated externally by humans (Strehl, 2002). For this reason, in this work
we will make use of external evaluation criteria solely. However, there exist some recent
efforts that aim to find correlations between internal and external cluster validity indices,
such as (Ingaramo et al., 2008). Nevertheless, for further insight on internal and relative
validity indices, see (Halkidi, Batistakis, and Vazirgiannis, 2002a; Halkidi, Batistakis, and
Vazirgiannis, 2002b; Maulik and Bandyopadhyay, 2002).
Therefore, evaluation will consist in testing whether the clustering solution reflects the
true group structure of the data set, captured in a reference clustering or ground truth4 .
A further advantage of this evaluation approach lies in the fact that external evaluation
measures can be used to compare fairly the performance of clustering algorithms regardless
of their foundations, as they make no assumption about the mechanisms used for finding
the clusters (Strehl, 2002).
There exist multiple ways of comparing a clustering solution to a ground truth. Quite
obviously, different approaches must be followed depending on the nature of the clustering
solution (i.e. whether it is hard or soft, hierarchical or partitional).
As regards the evaluation of soft clustering solutions, the main difficulty lies not in the
comparison process (see (Gopal and Woodcock, 1994; Jäger and Benz, 2000) for some classic
approaches), but in the creation of a fuzzy ground truth, which may require applying an
averaging scheme that accounts for systematic biases in the answers of the expert labelers
(Jäger and Benz, 2000; Jomier, LeDigarcher, and Aylward, 2005).
As far as the validation hierarchical clustering solutions is concerned, it is necessary to
use a hierarchical taxonomy as the ground truth. However, this type of ground truth is
4 The unsupervised nature of clustering makes that the performance of clustering algorithms cannot be
judged with the same certitude as for supervised classifiers, as the external categorization (ground truth)
might not be optimal. For instance, the way web pages are organized in the Yahoo! taxonomy is possibly not
the best structure possible, but achieving a grouping similar to the Yahoo! taxonomy is certainly indicative
of successful clustering (Strehl, 2002).
16