TESI DOCTORAL - La Salle

Chapter 2. Cluster ensembles and consensus clustering
consensus function (Goder and Filkov, 2008), which obtains the consensus partition by
conducting an iterative pivoting on the object dissimilarity matrix.
2.2.4 Consensus functions based on categorical clustering
A different approach to consensus clustering is the one related to categorical clustering,
which basically consists of transforming the contents of the cluster ensemble into quantitative
features that represent the objects, for subsequently clustering them according to this
novel representation —thus obtaining the consensus partition. The QMI (Quadratic Mutual
Information) consensus function of (Topchy, Jain, and Punch, 2003) posed the problem of
combining the partitions contained in a hard cluster ensemble in an information theoretic
framework, and consists of applying the k-means clustering algorithms on this new feature
space, which forces the user to set the desired number of clusters k in advance.
In (Punera and Ghosh, 2007), a novel fuzzy consensus function based on the Information
Theoretic K-means (ITK) algorithm was presented. Its rationale follows a similar approach
to that of (Topchy, Jain, and Punch, 2003). In this case, though, consensus clustering
is conducted on soft cluster ensembles (i.e. the compilation of the outcomes of multiple
fuzzy clustering processes), so each object in the data set is represented by means of the
concatenated posterior cluster membership probability distributions corresponding to each
one of the l fuzzy partitions in the cluster ensemble. Thus, using the Kullback-Leibler
divergence (KLD) between those probability distributions as a measure of the distance
between objects, the k-means algorithm is applied so as to obtain the consensus clustering
solution. Note that the ITK consensus function is capable of combining fuzzy partitions
with variable number of clusters, while producing a crisp consensus clustering solution.
Moreover, this consensus function allows assigning distinct weights to each clustering in
the cluster ensemble, which can be useful for the user to express his/her confidence on the
quality of some individual clusterings.
2.2.5 Consensus functions based on probabilistic approaches
Consensus clustering has also been approached from a probabilistic perspective. One of
the pioneering works in this direction was the Expectation-Maximization (EM) consensus
function proposed in (Topchy, Jain, and Punch, 2004), where a probabilistic model of the
consensus clustering solution is defined in the space of the contributing clusters. Such
model is based on a finite mixture of multinomial distributions, each component of which
corresponds to a cluster of the combined clustering, which is obtained as the solution to
the maximum likelihood problem solved by means of the EM algorithm. Contrasting with
other consensus functions, the authors highlight the low computational complexity of the
proposed method and its ability to combine partitions with different numbers of clusters.
Another probabilistic approach to the consensus clustering problem was presented in
(Long, Zhang, and Yu, 2005). The central matter in that work was finding a solution to
the cluster correspondence problem (which, as mentioned earlier, is due to the symbolic
identification of clusters caused by the unsupervised nature of the clustering problem). In
particular, the goal was to derive a correspondence matrix that desambiguates the clusters
of each individual clustering in the cluster ensemble (represented as a probabilistic or binary
membership matrix depending on whether the cluster ensemble is soft or hard) with regard
39