TESI DOCTORAL - La Salle

3.3.4 Experiments
Chapter 3. Hierarchical consensus architectures
This section presents the results of multiple experiments oriented to illustrate the computational
efficiency of DHCA, as well as to evaluate the predictive power of the running time
estimation methodology of table 3.6. Their design follows the scheme presented next.
Experimental design
– What do we want to measure?
i) The time complexity of deterministic hierarchical consensus architectures.
ii) The ability of the proposed methodology for predicting the computationally optimal
DHCA variant, in both the fully serial and parallel implementations.
iii) The predictive power of the proposed methodology based on running time estimation
vs the computational optimality criterion based on designing the DHCA
according to a decreasing diversity factor cardinality order, in both the fully
serial and parallel implementations.
– How do we measure it?
i) The time complexity of the implemented serial and parallel DHCA variants is
measured in terms of the CPU time required for their execution —serial running
time (SRTDHCA) and parallel running time (PRTDHCA).
ii) The estimated running times of the same DHCA variants –serial estimated running
time (SERTDHCA) and parallel estimated running time (PERTDHCA)– are
computed by means of the proposed running time estimation methodology, which
is based on the measured running time of c = 1 consensus clustering process. Predictions
regarding the computationally optimal DHCA variant will be successful
in case that both the real and estimated running times are minimized by the
same DHCA variant, and the percentage of experiments in which prediction is
successful is given as a measure of its performance. In order to measure the
impact of incorrect predictions, we also measure the execution time differences
(in both absolute and relative terms) between the truly and the allegedly fastest
DHCA variants in the case prediction fails. This evaluation process is replicated
for a range of values of c ∈ [1, 20], so as to measure the influence of this factor
on the prediction accuracy of the proposed methodology.
iii) Both computationally optimal DHCA variants prediction approaches are compared
in terms of the percentage of experiments in which prediction is successful,
and in terms of the execution time overheads (in both absolute and relative terms)
between the truly and the allegedly fastest DHCA variants in the case prediction
fails.
– How are the experiments designed? The f! DHCA variants corresponding to
all the possible permutations of the f diversity factors employed in the generation
of the cluster ensemble have been implemented (see table 3.6). As described in appendix
A.4, cluster ensembles have been created by the mutual crossing of f =3
diversity factors: clustering algorithms (dfA), object representations (dfR) and data
dimensionalities (dfD). Thus, in all our experiments, the number of DHCA variants is
75