CRANFIELD UNIVERSITY Eleni Anthippi Chatzimichali ...

Every kernel is characterised by a set of parameters – the hyperparameters – that have
to be optimised for a particular problem (Chapelle and Vapnik, 2000; Xu et al., 2006).
The Gaussian Radial Basis Function (RBF) kernel is particularly popular especially in
cases where there is little or no knowledge about the data under study. In RBF SVMs,
only one kernel parameter has to be optimised – the value of or – in addition to
the regularisation parameter C.
The value determines the degree of nonlinearity or width of the RBF kernel
(Boardman and Trappenberg, 2006; Verplancke et al., 2008), and is inversely related
to , the spread of the data, where . Higher values of result in greater
nonlinearity of the decision boundaries. More specifically, very high values of (low
values of ) potentially result in sharp peaks, “spiky” functions and boundaries that
surround individual samples as illustrated in Figure 1-6 (Valentini and Dietterich,
2004; Brereton, 2009). As the value decreases, the Gaussians become broader with
smoother surfaces that fit the data quite well. According to Keerthi and Lin (2003),
for small values of
the RBF kernel tends towards a linear boundary
(Boser et al., 1992; Hsu et al., 2003). Thus, a linear classifier may be considered a
special case of the RBF model since “with a suitable combination of hyperparameters
, the testing accuracy of the RBF kernel is at least as good as the linear kernel”
(Boser et al., 1992; Keerthi and Lin, 2003; Hsu et al., 2003; Chang et al., 2010).
In addition, as presented in Section 1.5.2.2, the cost parameter controls the
complexity of the SVM boundaries. More specifically, according to Xu et al. (2006),
the cost parameter controls the optimal trade-off between the two criteria of Equation
14, maximising the margin and minimising the training error. As , the hard
margin case is obtained, and thus, lower tolerance of misclassification is allowed
(Brereton, 2009). The high values of will force the creation of extremely complex
boundaries that misclassify as few training samples as possible. Large values of
may often lead to instances of overfitting (Foody and Mathur, 2004). On the contrary,
a lower value of creates wider margins, which allows instances close to the
boundary to be ignored (Ben-Hur et al., 2010). For very low values of , independent
of the value, the SVM models are unable to learn, causing a problem of underfitting
(Valentini and Dietterich, 2004).
20