Theory of Statistics - George Mason University

778 0 Statistical Mathematics
Now, consider two eigenvalues ci = cj, that is, an eigenvalue of multiplicity
greater than 1 and distinct associated eigenvectors vi and vj. By what we
just saw, an eigenvector associated with ck = ci is orthogonal to the space
spanned by vi and vj. Assume vi is normalized and apply a Gram-Schmidt
transformation to form
˜vj =
1
vj − 〈vi, vj〉vi (vj − 〈vi, vj〉vi),
yielding a vector orthogonal to vi. Now, we have
1
A˜vj =
vj − 〈vi, vj〉vi (Avj − 〈vi, vj〉Avi)
1
=
vj − 〈vi, vj〉vi (cjvj − 〈vi, vj〉civi)
1
= cj
vj − 〈vi, vj〉vi (vj − 〈vi, vj〉vi)
= cj˜vj;
hence, ˜vj is an eigenvector of A associated with cj. We conclude therefore that
the eigenvectors of a symmetric matrix can be chosen to be orthogonal.
A symmetric matrix is orthogonally diagonalizable, and because the eigenvectors
can be chosen to be orthogonal, and can be written as
where V V T = V T V = I, and so we also have
A = VCV T , (0.3.4)
V T AV = C. (0.3.5)
Such a matrix is orthogonally similar to a diagonal matrix formed from its
eigenvalues.
Spectral Decomposition
When A is symmetric and the eigenvectors vi are chosen to be orthonormal,
I =
, (0.3.6)
so
i
A = A
=
i
i
viv T i
viv T i
Aviv T i
Theory of Statistics c○2000–2013 James E. Gentle
=
civiv T i . (0.3.7)
i