CRANFIELD UNIVERSITY Eleni Anthippi Chatzimichali ...

2.3.2 Classification Results
2.3.2.1 Overall Accuracies (%CC)
The prediction results of all implemented classification models are illustrated in the
bar charts of Figure 2-6 as percentages of correctly classified samples ( ). The
first model to be tested on raw standalone data was PLS-DA with LOOCV, which
constitutes a single classifier, and not an ensemble of classifiers. According to Section
1.6.3, LOOCV is a nearly unbiased, albeit with a high variance, technique that may
often lead to misleading results (Efron, 1983; Kohavi, 1995; Duan et al., 2003;
Glasmachers, 2008); in this instance, since the test set is used for both model
construction and optimisation purposes, it may often lead to over-optimistic results.
For case study 1, the overall accuracies of FTIR, HPLC and e-nose are equal to 63%,
84% and 59% respectively. Indeed, when the afore-mentioned percentages are
compared to the accuracies of the other classification models of Figure 2-6, they
appear to be overly optimistic.
In the case of the classification ensembles, HPLC clearly demonstrates the highest
overall accuracies among the three instrumental techniques. The HPLC data appear to
generate higher in the case of SVMs, especially for linear SVMs, compared to
PLS-DA. Even though PLS-DA and linear SVMs both construct linear decision
boundaries, the difference in accuracies can be possibly explained by the fact that
support vector machines conduct linear classification based on a different approach
than PLS-DA. As stated in Brereton et al. (2009), the SVM boundary depends solely
on the selected support vectors, while the remaining samples have no influence over
it. On the contrary, methods such as PLS-DA use all available samples in order to
determine the separating planes between the classes (Xu et al., 2006). Finally, since
the results of both linear and nonlinear (RBF) SVM ensembles approximate 80%, it
can be concluded that the boundaries of the RBF SVMs may have been nearly linear,
but still retain a wide margin (Brereton et al., 2009); as stated in Section 1.5.2.3,
“with a suitable combination of hyperparameters , the testing accuracy of the
RBF kernel is at least as good as the linear kernel” (Boser et al., 1992; Keerthi and
Lin, 2003; Hsu et al., 2003; Chang et al., 2010).
47