TESI DOCTORAL - La Salle

4.2. Flat vs. hierarchical self-refining
Consensus Consensus function
solution CSPA EAC HGPA MCLA ALSAD KMSAD SLSAD
non-refined 0.011 0.011 0.026 0.638 0.009 0.011 0.029
best non/self-refined 0.004 0.019 0.005 0.002 0.002 0.001 0.006
Table 4.8: φ (NMI) variance of the non-refined and the best non/self-refined consensus clustering
solutions across the flat, RHCA and DHCA consensus architectures, averaged across
the twelve data collections.
Consensus Consensus function
architecture CSPA EAC HGPA MCLA ALSAD KMSAD SLSAD
flat 30.4 50 53.1 11 25 23.8 37.5
RHCA 25 35.9 38.4 3.9 24.4 24.1 36.6
DHCA 12.5 42 40.1 17 0 29.5 37.5
Table 4.9: Percentage of experiments in which the supraconsensus function selects the top
quality consensus clustering solution, averaged across the twelve data collections.
we have assumed the use of the top quality self-refined consensus clustering solution. Quite
obviously, achieving the encouraging results reported would require using a supraconsensus
function that, in an automatic manner, would detect that best self-refined consensus clustering
in any given situation. The next section is devoted to the performance analysis of
such supraconsensus function.
4.2.2 Evaluation of the supraconsensus process
As regards the performance of the supraconsensus function proposed by (Strehl and Ghosh,
2002), we have firstly evaluated the percentage of experiments in which the supraconsensus
function selects the highest quality consensus clustering solution. Table 4.9 presents the
results averaged across all the data collections, for each consensus function and architecture.
The average accuracy with which the supraconsensus function selects the top quality selfrefined
consensus selection is 29%, i.e. it manages to select the best solution in less than a
third of the experiments conducted.
This somehow contradicts the beautiful conclusions of (Strehl and Ghosh, 2002), where
φ (ANMI) (E, λc) is presented as a suitable surrogate of φ (NMI) (γ, λc) for selecting the best
consensus clustering solutions in real scenarios, where a ground truth γ is not available.
Such conclusion was supported by the fact that both φ (ANMI) (E, λc) andφ (NMI) (γ, λc)
follow very similar patterns as regards their growth (i.e. the higher φ (NMI) (γ, λc), the
higher φ (ANMI) (E, λc)). However, such claims were sustained on experiments using synthetic
clustering results. In several of our experiments, in contrast, we have witnessed that
this behaviour is not always obeyed in a strict fashion.
Just for illustration purposes, we have conducted a toy experiment, in which a set of
randomly picked 300 cluster ensemble components corresponding to the Zoo data collection
have been evaluated in terms of i) their φ (NMI) with respect to the ground truth, and ii)
their φ (ANMI) with respect to the 299 remaining clusterings selected. Figure 4.2 depicts
both magnitudes, where the horizontal axis of each figure corresponds to an index of the
clusterings in the ensemble arranged in decreasing order of φ (NMI) with respect to the
120