TESI DOCTORAL - La Salle

Chapter 7. Conclusions
clustering results —although in other contexts, such as jointly multimodal data analysis
and synthesis, it becomes a crucial process (Sevillano et al., 2009).
For this reason, the key point of our approach to robust multimedia clustering consists
in not prioritizing nor discarding any of the modalities. Rather the contrary, the user is
encouraged to create clusterings upon each separate modality and on feature level fused
modalities, compiling them all into a multimodal cluster ensemble, upon which a consensus
clustering is created.
Interestingly enough, the application of this strategy –which is nothing but a generalized
version of our approach to robust clustering– naturally calls for the use of hierarchical
consensus architectures, as the existence of multiple (say m) modalities increases cluster
ensemble sizes by a minimum factor of m + 1 (as we consider the m original object representations
plus the one created by their feature level fusion), which poses a computational
challenge to the execution of flat consensus clustering. Furthermore, the hypothetical inclusion
of low quality components in such a large cluster ensemble makes the application of
self-refining procedures an attractive alternative for obtaining good consensus clusterings
upon the aforementioned multimodal cluster ensemble.
In order to evaluate the effect of multimodality in a modular manner, separate consensus
processes have been conducted for each original data modality and for the modality
derived from the early fusion of these. To that effect, a deterministic hierarchical consensus
architecture has been employed in our multimodal consensus clustering experiments, as it
allows a structured construction of consensus clusterings both within and across modalities.
As regards within modality consensus, the results obtained reveal that consensus clusterings
obtained on the multimodal modality (i.e. the one resulting from the early fusion
of the original modalities of the data) attain higher φ (NMI) scores than their unimodal
counterparts in an average 56% of the experiments conducted.
When the unimodal and multimodal consensus clusterings are combined –thus giving
rise to intermodal consensus clusterings– we observe that, in terms of φ (NMI) with respect
to the ground truth, these are better than the unimodal ones in a 59.5% of the experiments,
while this percentage is 34.7% when compared to the multimodal consensus clustering.
However, the fairly distinct results obtained depending on the data set and consensus
function employed suggest that creating an intermodal consensus clustering is a pretty
generic way of proceeding, as its eventual inferior quality can be compensated by means of
a subsequent self-refining procedure followed by an unsupervised supraconsensus selection
of the final consensus clustering.
If the maximum and median quality components of the multimodal cluster ensemble
are taken as reference thresholds for evaluating the robustness of the self-refined consensus
clustering selected by supraconsensus, we observe that it is a 36.6% worse than the former
and a 93.5% better than the latter (measured in relative percentage φ (NMI) variations). As in
the unimodal case, this performance would be improved if a better supraconsensus selection
process was devised —which, as aforementioned, is one of the future work priorities.
As regards the future research lines in the multimodal clustering area, we plan to investigate
early multimodal fusion techniques capable of unveiling constructive interactions
between modalities, besides applying selection based consensus self-refining on the multimodal
cluster ensemble, as we conjecture that will probably yield higher quality clusterings
than those obtained by consensus based self-refining.
199