TESI DOCTORAL - La Salle

Chapter 5
Multimedia clustering based on
cluster ensembles
As already outlined in section 1.3, multimodality is an increasingly noticeable trend as
far as the nature of data is concerned. Given the growing ubiquity of multimedia data,
it seems logical to consider that the derivation of robust clustering strategies as a means
for organizing the increasingly larger multimodal repositories available is already a field of
interest in itself.
However, it is important to take into account that the direct application of classic
clustering algorithms for partitioning multimedia data collections may turn out to be suboptimal.
The reason for this is twofold: firstly, the usual indeterminacies that condition
the performance of clustering algorithms are multiplied due to the existence of several data
modalities. This means that the user must not only make a decision regarding which is
the object representation or the clustering algorithm that are supposed to yield the best
partition of the data —i.e. the ones best describing the natural group structure of the
data. Furthermore, it is also necessary to make a decision regarding on which of the m
data modalities clustering is to be conducted, as classic clustering algorithms are designed
to operate on unimodal data.
And secondly, notice that clustering multimedia data using a single modality entails
ignoring the presumable positive synergies that may exist between the different modalities,
which could be of interest for deriving a better partition of the data. The only way a
classic clustering algorithm can take advantage of the possible benefits of multimodality
consists in creating multimodal representations of the objects, conducting an early fusion
of the features corresponding to distinct modalities prior to clustering. That is, clustering
is conducted on a single, artificially generated multimodal representation of the objects
created by the combination of the m original modalities. However, feature fusion may be
benefitial or not as regards the quality of the clustering results, as reported in appendix
B.2, which turns the application of this strategy into a further indeterminacy to deal with.
For these reasons, in this chapter we propose applying consensus clustering as a means
for clustering multimedia data robustly, as it provides a natural way for combining i) the
results of clustering processes run on each one of the m distinct modalities —thus conducting
a late fusion of modalities, and ii) the partitions obtained upon the multimodal data
representation derived by the early fusion of the features of the m modalities. By doing so,
133