CRANFIELD UNIVERSITY Eleni Anthippi Chatzimichali ...

1.5 Multivariate Analysis: Supervised Learning
This research will solely focus on the investigation of multivariate classification
techniques. A classifier, also known as predictor, can be defined as “a function that
maps an unlabelled instance to a label using internal data structures” (Kohavi, 1995).
Supervised classification derives from the concept of learning by experience (Ciosek
et al., 2005). A model is trained to distinguish groups of a predefined dataset where
the class of each sample is already known. The training dataset is used to establish a
mathematical model, which in turn should be capable of predicting the class
membership of ideally unseen data (Izenman 2008). Supervised learning algorithms
are characterised by a predefined set of parameters, which may have a profound effect
on the resulting performance (Chapelle et al., 2002). Therefore, thorough selection of
these parameters is a necessity.
1.5.1 Partial Least Squares – Discriminant Analysis
Partial Least Squares-Discriminant Analysis (PLS-DA) (Barker and Rayens, 2003)
is a widely used classification technique in the field of chemometrics (Westerhuis et
al., 2008). It is a linear model that consists of Partial Least Squares (PLS) (Wold,
1975) dimensionality reduction and Linear Discriminant Analysis (LDA) applied on
the PLS components. Unlike PCA, which attempts to capture the maximum variance,
PLS-DA aims to maximise the covariance – accomplish both correlation and
maximum variance – between the input data and an output class (Wise et al., 2003;
Weber et al., 2011).
In matrix notation, suppose that is a predictor matrix, which corresponds to
independent variables, and is a class affiliation vector that holds the dependent
variables. PLS-DA attempts to model the relationship between dependent and
independent variables by projecting the data matrices and into a new subspace.
The orthogonal axes in the PLS subspace are also known as Latent Variables (LVs).
The output of PLS-DA is the product of two smaller matrices, the scores matrix
(PLS-DA scores) and the predicted affiliation matrix. Thus, it satisfies the
mathematical equations
14