Principal Component Analysis (PCA)

The PCA Solution
• The desired direction of maximum variance is the
direction of the unit-length eigenvector v1 of Cx with the
largest eigenvalue
• Next, we want to find the direction whose projection
has the maximum variance among all directions
perpendicular to the first eigenvector v1 . The solution to
this is the unit-length eigenvector v2 of Cx with the 2nd highest eigenvalue.
• In general, the k-th direction whose projection has the
maximum variance among all directions perpendicular to
the first k-1 directions is given by the k-th unit-length
eigenvector of Cx . The eigenvectors are ordered such
that:
λ ≥
1
≥ λ2
≥ Lλn−1
• We say the k-th principal component of x is given by:
y k =
Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 8
v T
k
x
λ
n