Applied Statistics Using SPSS, STATISTICA, MATLAB and R

8.2 Dimensional Reduction
8.2 Dimensional Reduction 337
When using principal component analysis for dimensional reduction, one must
decide how many components (and corresponding variances) to retain. There are
several criteria published in the literature to consider. The following are commonly
used:
1. Select the principal components that explain a certain percentage (say, 95%)
of tr(Λ). This is a very simplistic criterion that is not recommended.
2. The Guttman-Kaiser criterion discards eigenvalues below the average
tr(Λ)/d (below 1 for standardised data), which amounts to retaining the
components responsible for the variance contributed by one variable if the
total variance was equally distributed.
3. The so-called scree test uses a plot of the eigenvalues (scree plot),
discarding those starting where the plot levels off.
4. A more elaborate criterion is based on the so-called broken stick model. This
criterion discards the eigenvalues whose proportion of explained variance is
smaller than what should be the expected length lk of the kth longest
segment of a unit length stick randomly broken into d segments:
l
k
1
=
d
d
∑
i=
k
1
. 8.12
i
A table of lk values is given in Tools.xls.
5. The Bartlett’s test method is based on the assessment of whether or not the
null hypothesis that the last p − q eigenvalues are equal, λq+1 = λq+2 = …
= λp, can be accepted. The mathematics of this test are intricate (see Jolliffe
IT, 2002, for a detailed discussion) and its results often unreliable. We pay
no further attention to this procedure.
6. The Velicer partial correlation procedure uses the partial correlations
among the original variables when one or more principal components are
removed. Let Sk represent the remaining covariance matrix when the
covariance of the first k principal components is removed:
k
S S − λ u u ’
; k = 0,
1,
K,
d . 8.13
k = ∑
i=
1
i
i
i
Using the diagonal matrix Dk of Sk, containing the variances, we compute
the correlation matrix:
−1/
2 −1/
2
k = D k S k D k
R . 8.14
Finally, with the elements rij(k) of Rk we compute the following quantity: