TESI DOCTORAL - La Salle

1.2. Clustering in knowledge discovery and data mining
weighted graph whose edges reflect the similarity between each pair of objects. This
makes graph based clustering conceptually similar to hierarchical clustering (Jain and
Dubes, 1988), a good example of which is the Chameleon hierarchical clustering algorithm,
which is based on the k-nearest neighbour graph (Karypis, Han, and Kumar,
1999). Other (non-hierarchical) graph based clustering algorithms are i) Zahn’s algorithm
(Zahn, 1971), which is based on discarding the edges with the largest lengths
on the minimal spanning tree so as to create clusters, ii) CLICK, based on computing
the minimum weight cut to form clusters (Sharan and Shamir, 2000), and iii)
MajorClust, which is based on the weighted partial connectivity of the graph, a measure
whose maximization implicitly determines the optimum number of clusters to be
found (Stein and Niggemann, 1999). Lastly, the more recent family of spectral clustering
algorithms can be included in the context of graph based clustering. This type
of algorithms are often reported as outperforming traditional clustering techniques
(von Luxburg, 2006). In addition, spectral clustering is simple to implement and can
be solved efficiently by standard linear algebra methods, as, in short, it boils down to
computing the eigenvectors of the Laplacian matrix of the graph. Several variants of
spectral clustering algorithms have been proposed, differing in the way the similarity
graph and the Laplacian matrix are computed (e.g. (Shi and Malik, 2000; Ng, Jordan,
and Weiss, 2002)).
4. Clustering based on combinatorial search: this approach is based on considering clustering
as a combinatorial optimization problem that can be solved by applying search
techniques for finding the global (or approximate global) optimum clustering solution.
In this context, two paradigms have been followed in the design of clustering algorithms:
stochastic optimization methods and deterministic search techniques. Among
the former, some of the most popular approaches are based on evolutionary computation
(e.g. hard or soft clustering based on genetic algorithms (Hall, Özyurt,
and Bezdek, 1999; Tseng and Yang, 2001)), simulated annealing (e.g. (Selim and
Al-Sultan, 1991)), Tabu search (Al-Sultan, 1995) and hybrid solutions (Chu and Roddick,
2000; Scott, Clark, and Pham, 2001), whereas deterministic annealing is the most
typical deterministic search technique applied to clustering (Hofmann and Buhmann,
1997).
5. Clustering based on neural networks: the well-known learning and modelling abilities
of neural networks have been exploited to solve clustering problems. The two most
successful neural networks paradigms applied to clustering are i) competitive learning,
where the Self-Organizing Maps (Kohonen, 1990) and Generalized Learning Vector
Quantization (Karayiannis et al., 1996) play a salient role, and ii) adaptive resonance
theory (Carpenter and Grossberg, 1987), which encompasses a whole family of neural
networks architectures that can be used for hierarchical (Wunsch et al., 1993) and
soft (Carpenter, Grossberg, and Rosen, 1991) clustering.
6. Kernel based clustering: the rationale of kernel-based learning algorithms is simplifying
the task of separating the objects in the data set by nonlinearly transforming them
into a higher-dimensional feature space. Through the design of an inner-product kernel,
the time-consuming and sometimes even infeasible process of explicitly describing
the nonlinear mapping and computing the corresponding data points in the transformed
space can be avoided (Xu and Wunsch II, 2005). A recent example of this
approach is Support Vector Clustering (SVC) (Ben-Hur et al., 2001), that employs
14