TESI DOCTORAL - La Salle

Chapter 5. Multimedia clustering based on cluster ensembles
Moreover, suppose that our multimedia data collection is composed of m modalities.
That is, each of the n objects is simultaneously represented by m disjoint sets of real-valued
attributes –each one of which corresponds to one of the m modalities– of sizes d1, d2, ...,
dm, sothatd1 + d2 + ... + dm = d. Thus, the multimodal data set matrix X can be
decomposed in m submatrices X1, X2, ..., Xm. Each one of these matrices Xi (of size
di × n, ∀i ∈ [1,m]) represents all the objects in the data set according to each one of the
m modalities it contains —see figure 5.1.
Given this scenario, a first subset of the clusterings that will constitute the multimodal
cluster ensemble E are generated through the application, upon each 1 submatrix
Xi, (∀i ∈ [1,n]), of f mutually crossed diversity factors dfj, ∀j ∈ [1,f]. If the same set of
diversity factors is applied on the m modalities, the number of clusterings generated in this
first subset is equal to:
l1 mod = m|df1||df2| ...|dff| (5.1)
where the |·|operator denotes the cardinality of a set.
Secondly, another subset of clusterings is created by the application of a set of diversity
factors (not necessarily equal to the previous one) upon a fused multimodal representation
of the data set. This representation can be generated by means of any early feature fusion
process, such as the application of a projection-based object representation technique on
the d-dimensional vectors resulting from the concatenation of the features corresponding
to the m modalities (La Cascia, Sethi, and Sclaroff, 1998; Zhao and Grosky, 2002; Benitez
and Chang, 2002; Snoek, Worring, and Smeulders, 2005; Gunes and Piccardi, 2005). This
second subset of clusterings will be referred to using the symbol m mod, astheyareobtained
upon an object representation that combines the m modalities into a single one.
Assuming, for simplicity, that the same set of diversity factors are employed for creating
the subsets of unimodal and multimodal clusterings, the number of multimodal partitions
created becomes:
lm mod = |df1||df2| ...|dff | (5.2)
Finally, the mere compilation of the unimodal and multimodal partitions constitute the
multimedia cluster ensemble E, the size of which is equal to:
l = l1 mod + lm mod =(m +1)|df1||df2| ...|dff | (5.3)
As regards the creation of the multimodal cluster ensembles E on the four multimedia
data collections employed in this work (see appendix A.2.2 for a description), three diversity
factors have been applied: clustering algorithms (dfA), object representations (dfR)
and object representation dimensionalities (dfD). In the following paragraphs, a detailed
description of these diversity factors and their role in the cluster ensemble creation process
are presented.
Starting with the original object features, which constitute the baseline representation,
additional object representations are derived by means of feature extraction based on
1 As these clusterings are created by running multiple clustering processes separately on each modality,
we refer to them using the symbol 1 mod, which stands for “one modality”.
135