TESI DOCTORAL - La Salle

3.1. Motivation
An additional and very relevant point as regards the computational efficiency of hierarchical
consensus architectures is that they naturally allow the parallel execution of the
consensus clustering processes of every HCA stage —quite obviously, this will ultimately
depend on the availability of computing resources. Thus, the degree of parallelism in executing
the consensus of every HCA stage will set the lower and upper bounds of the time
required for obtaining the final consensus clustering λc.
In the best-case scenario, the HCA running time can be as low as the sum of the
execution times of the longest-lasting consensus task of each stage of the architecture,
provided that the available computational resources allow the parallel computation of all
the intermediate consensus solutions of any given stage.
On the contrary, if the execution of the halfway consensus is serialized, the time required
for running the whole HCA amounts to the sum of the execution times of all the consensus
processes of the stages of the hierarchical consensus architecture, which constitutes the
upper bound of the running time of a hierarchical consensus architecture.
Therefore, depending on the design of the HCA, the simultaneously available computing
resources and the characteristics of the data set, structuring the consensus clustering task
in a hierarchical manner may be more or less computationally beneficial (or not beneficial
at all) as compared to its flat counterpart. From a practical viewpoint, our general idea is
to provide the user with simple tools that, for a given consensus clustering problem, allow
to decide a priori whether hierarchical consensus architectures are more computationally
efficient than traditional flat consensus and, if so, implement the HCA variant of minimal
complexity.
Moreover, it is important to highlight the fact that, in cases where the flat execution
of the consensus function F becomes impossible due to memory limitations caused by the
large size of the cluster ensemble, a carefully designed HCA will allow obtaining a consensus
clustering solution.
Let us now elaborate briefly on several notational definitions regarding hierarchical
consensus architectures that will be of help when describing our proposals in detail. We
suggest the reader resort to the generic HCA topology depicted in figure 3.1(b) for a better
understanding of the concepts we are about to expose.
Firstly, a hierarchical consensus architecture is structured in s successive stages. The
number of intermediate consensus solutions obtained at the output of the ith stage is denoted
as Ki —notice that Ks = 1 (i.e. the last stage yields the single final consensus
clustering solution λc). The notation used for designating the jth halfway consensus clustering
created at the ith HCA stage is λ i cj ,wherei∈ [1,s− 1] and j ∈ [1,Ki].
Another important factor in the definition of HCAs is the size of the mini-ensembles,
which may vary from stage to stage or even within the same stage. For this reason, we
denote as bij the size of the mini-ensemble upon which the jth consensus process of the ith
HCA stage is conducted. Notice that bs1 = Ks−1 (i.e. the last consensus stage combines
all the intermediate clusterings output by the previous stage into the single final consensus
clustering solution λc), while, in the HCA presented in figure 3.1(b), b1j =2∀j ∈ [1,K1].
Moreover, notice that hierarchical architectures naturally allow the use of distinct consensus
functions across the distinct stages (or even within the same stage). However, in
this work we assume that a single consensus function F is applied for conducting all the
consensus processes involved.
48