TESI DOCTORAL - La Salle

5.2. Self-refining multimodal consensus architecture
Data set Modality df D range |df D|
audio [30,120] 10
CAL500
text [30,70] 5
audio + text [30,200] 18
speech [100,600] 11
IsoLetters image [3,16] 14
speech + image [100,600] 11
object [60,120] 7
InternetAds collateral [100,1000] 19
object + collateral [100,1000] 19
image [50,350] 7
Corel
text [100,450] 8
image + text [100,800] 15
Table 5.1: Range and cardinality of the dimensional diversity factor dfD per modality for
each one of the four multimedia data sets.
Principal Component Analysis, Independent Component Analysis, Random Projection and
Non-negative Matrix Factorization (this last representation can only be applied when the
original object features are non-negative). Thus, the total number of object representationsiseither|dfR|
= 4 (for the CAL500 and IsoLetters collections) or |dfR| = 5 (for the
InternetAds and Corel data sets). It is important to notice that these feature extraction
based object representations are derived for each one of the m = 2 modalities these data
sets contain, plus for the multimodal baseline representation created by concatenating the
features of both modes.
For each feature extraction based object representation and modality, a set of distinct
representations are created by conducting a sweep of dimensionalities, which constitutes
the second diversity factor, dfD. Quite obviously, its cardinality depends on the data set
and modality. The range and cardinality of dfD per modality corresponding to each one
of the four multimedia data sets employed in the experimental section of this chapter are
presented in table 5.1.
And finally, the clusterings that make up the multimodal cluster ensemble E are created
by running |dfA| = 28 clustering algorithms from the CLUTO clustering package
(see appendix A.1) on each distinct object representation. As a result, a total of l =
2856, 3108, 5124 and 3444 partitions are obtained for the CAL500, IsoLetters, InternetAds
and Corel multimodal data collections, respectively. Notice that, in our case, diversity
factors are not mutually crossed, as the baseline object representations lack dimensional
diversity. Therefore, the generic expressions of equations (5.1) to (5.3) do not apply in our
case.
5.2 Self-refining multimodal consensus architecture
Once the multimodal cluster ensemble E is built, the next step consists in deriving a consensus
clustering solution λc upon it. Recall that, according to the conclusions drawn in
chapter 3, it may be more computationally efficient to tackle this task by means of a flat or
a hierarchical consensus architecture depending on the size of the cluster ensemble.
136