CRANFIELD UNIVERSITY Eleni Anthippi Chatzimichali ...

3.3.2 Nonlinear Models
In order to test the performance and the applicability of the new heuristic for
real-world cases, a simulation based on the methodology presented thus far was
conducted using the HPLC dataset. The simulation aims towards the comparison
between the grid-search and the Box complex algorithm for the optimisation of
ensembles of nonlinear SVMs (RBF) via bootstrapping. In order for all results to be
directly comparable, the exact same train, test and validation datasets were used for
both algorithms. The algorithms are assessed based on their average train and test
accuracies as well as the execution times.
In Section 2.2.4 a combination of a coarse grid-search followed by a finer grid-search
was proposed as a means of optimising the RBF hyperparameters . The
percentages of correctly classified samples ( ) of these models were presented in
Figure 2-6. In addition, a new ensemble of SVM classifiers was optimised and tested
using an exhaustive grid-based search with a refined resolution of , which
allowed greater grid granularity. The Box complex algorithm was also employed for
the minimisation of the average bootstrapping test error during the training process of
the SVMs. In this case, the inequality constraints correspond to the minimum and
maximum predefined value boundaries that were set for the RBF hyperparameters
( ) in Section 2.2.4, where { } and { }. The
formation of the initial complex begins with the selection of a random feasible point
that must satisfy the minimum and maximum hyperparameter constraints as presented
in Figure 3-5.
A classification model has been selected at random out of the ensemble of classifiers
(100 independent classifiers) for demonstrative purposes as a means of visually
assessing the optimisation outcome of the two techniques. Figure 3-5 illustrates stepby-step
the Box complex simplices towards finding the optimal combination of
hyperparameters. In addition, the high-resolution grid plot that constitutes the
background of each figure, derives from the grid-search optimisation with a resolution
equal to ; each grid point corresponds to an average training error of 100
independent bootstrap iterations for a predefined combination of hyperparameters.
68