CRANFIELD UNIVERSITY Eleni Anthippi Chatzimichali ...

The first PC places all points the farthest apart from each other, extracting thus the
highest variance. Similarly, a perpendicular to the first PC axis is drawn for the
second PC, which accounts for the second highest variance. The process is repeated to
get multiple orthogonal principal components. Each successive orthogonal axis
displays a decreasing amount of the total variance.
Figure 1-4 Extracting the Principal Components
PCA projects the input data into a subspace of reasonable and meaningful dimension by setting new
directions in the pattern space. Thus, the projected cloud of points is as dispersed as possible. The
figure has been extracted from Kavraki (2007).
In matrix notation, suppose that variables have been observed on instances.
The generated multivariate dataset forms an data matrix with rows
(observations) and columns (variables). Thus, a cloud of points is created in an
-dimensional space, where a new axis is used per variable. The PCA algorithm
reduces the size of possibly correlated variables into new uncorrelated variables
(PCs), where . Each PC can be expressed mathematically as an orthogonal
linear combination of the original variables . In PCA, the original matrix
can be decomposed into the scores matrix , loadings matrix
and a residuals matrix . Several algorithms can be used for data decomposition, the
most widely applied of which are Singular Value Decomposition (SVD) and the
Nonlinear Iterative Partial Least Squares (NIPALS) (Wold, 1975) algorithm. In
general, the mathematical equation for PCA can be described by
Equation 3 PCA scores and loadings
11