CRANFIELD UNIVERSITY Eleni Anthippi Chatzimichali ...

TABLE OF FIGURES
Figure 1-1 Evolution from molecular biology to systems biology ................................. 2
Figure 1-2 Electromagnetic spectrum ............................................................................ 4
Figure 1-3 Representation of possible correlations and redundancies in highdimensional
data ................................................................................................... 10
Figure 1-4 Extracting the Principal Components ......................................................... 11
Figure 1-5 Nonlinear SVM classifier ........................................................................... 18
Figure 1-6 The effect of the hyperparameter on the SVM boundaries ...................... 21
Figure 1-7 k-fold Cross-Validation............................................................................... 25
Figure 1-8 Leave-One-Out Cross Validation (LOOCV) .............................................. 26
Figure 1-9 Model complexity and overfitting; the bias-variance trade-off .................. 28
Figure 1-10 Permutation tests and the P-value ............................................................. 29
Figure 2-1 Mean FTIR spectra for case study 1 in the fingerprint region (1500-1000
cm -1 ) ...................................................................................................................... 35
Figure 2-2 Sampling with Libra e-nose ........................................................................ 36
Figure 2-3 Data intersection ......................................................................................... 38
Figure 2-4 The process of constructing an ensemble of RBF SVMs optimised via
bootstrapping ........................................................................................................ 42
Figure 2-5 PCA score plots with 95% confidence ellipses for case study 1 ................ 46
Figure 2-6 Overall accuracies (%CC) for the standalone datasets of case study 1 ...... 49
Figure 2-7 Class prediction rates of the standalone (prior to PCA) datasets for case
study 1 .................................................................................................................. 52
Figure 2-8 Comparison of the optimisation of the hyperparameters of RBF SVMs via
bootstrapping, 10-fold cross-validation and LOOCV respectively ...................... 55
Figure 2-9 Three-dimensional error surface plots for the optimisation of the RBF
parameters ............................................................................................................. 56
Figure 3-1 Master/Slave architecture ........................................................................... 60
Figure 3-2 Embarrassingly parallel problems in the analysis pipeline ......................... 61
Figure 3-3 The steps of the Nelder-Mead algorithm .................................................... 64
Figure 3-4 The relationship between the number of slave processors (master/slave
model) and the execution times of an ensemble of PLS-DA with bootstrapping 67
Figure 3-5 Step-by-step representation of the Box complex algorithm towards
identifying the optimal hyperparameters and the minimum bootstrapping error
(HPLC data) .......................................................................................................... 70
Figure 3-6 Contour plots of the density estimation of the optimal hyperparameters as
defined by the Box complex algorithm ................................................................ 71
Figure 3-7 Density, filled-contour, grid and contour plots for the HPLC optimisation 72
Figure 3-8 Comparison of the prediction accuracies between the grid-Search and the
Box complex algorithm (HPLC data) ................................................................... 73
Figure 3-9 Histograms of the number of iterations and function evaluations
respectively for an ensemble of nonlinear (RBF) SVMs optimised using the Box
complex algorithm. ............................................................................................... 74
Figure 3-10 Comparison of the execution times for the tuning of a single RBF SVM
ensemble, when optimised with different techniques, via bootstrapping ............. 76
Figure 3-11 Comparison of the execution times of 100 permutation tests for the (RBF)
SVMs when optimised with different techniques, via bootstrapping ................... 77
viii