CRANFIELD UNIVERSITY Eleni Anthippi Chatzimichali ...

The “one-against-all” approach (Bottou et al., 1994) is the earliest and simplest
method proposed, which involves determining how well a sample is modelled by each
class individually, and subsequently selecting the class it is modelled-by at its best
(Foody and Mathur, 2004; Brereton and Lloyd, 2009). Thus, for a class problem,
binary classifiers are created and trained, one for each given class (Karatzoglou et al.,
2006). The “one-against-all” approach is based on a “winner-takes-all” strategy (Duan
and Keerthi, 2005). On the contrary, the most recent “one-against-one” (Kressel,
1999) approach constructs several binary SVM classifiers for each available pairwise
combination of classes (Hsu and Lin, 2002). Subsequently, the results of all individual
classifiers are aggregated using a voting mechanism such as “majority vote” (Duan
and Keerthi, 2005). In this case,
SVM models are created, one for each
pairwise combination of classes. According to Hsu and Lin (2002), this approach
verily generates robust outcome when employed with SVMs.
1.5.3 Ensemble Models
A major problem in multivariate classification is that often standalone classifiers may
achieve very high classification accuracies in the training process, however, their
generalisation performance (test performance) when applied to new unseen data may
greatly vary. Therefore, instead of using only a single final model, the concept of a
classification ensemble is based on the fusion of many diverse yet accurate models to
obit a range of predictions (Dietterich, 2000; Westerhuis et al., 2008). Thus, this
approach aims to improve the overall classification accuracy, and provide more stable
and accurate results. An ensemble can be constructed using any type of classifier such
as PLS-DA and SVMs.
22