A First Course in Linear Algebra, 2017a

7.4. Orthogonality 405
Theorem 7.66: Singular Value Decomposition
Let A be an m×n matrix. Then there exist orthogonal matrices U and V of the appropriate size such
that A = UΣV T where Σ is of the form
[ ] σ 0
Σ =
0 0
and σ is of the form
for the σ i the singular values of A.
σ =
⎡
⎢
⎣
⎤
σ 1 0
⎥ ... ⎦
0 σ k
Proof. There exists an orthonormal basis, {⃗v i } n i=1 such that AT A⃗v i = σ 2
i ⃗v i where σ 2
i > 0fori = 1,···,k,(σ i > 0)
and equals zero if i > k. Thus for i > k, A⃗v i =⃗0 because
For i = 1,···,k, define⃗u i ∈ R m by
Thus A⃗v i = σ i ⃗u i .Now
⃗u i •⃗u j = σ −1
i
A⃗v i • A⃗v i = A T A⃗v i •⃗v i =⃗0 •⃗v i = 0.
= σ −1
i
⃗u i = σ −1
i A⃗v i .
A⃗v i • σ −1 A⃗v j = σ −1 ⃗v i • σ −1 A T A⃗v j
j
⃗v i • σ −1
Thus {⃗u i } k i=1 is an orthonormal set of vectors in Rm .Also,
AA T ⃗u i = AA T σ −1
i
j
i
σ 2 j ⃗v j = σ j
σ i
(
⃗vi •⃗v j
)
= δij .
A⃗v i = σi
−1 AA T A⃗v i = σi −1 Aσi 2 ⃗v i = σi 2 ⃗u i.
Now extend {⃗u i } k i=1 to an orthonormal basis for all of Rm ,{⃗u i } m i=1 and let
U = [ ]
⃗u 1 ··· ⃗u m
while V =(⃗v 1 ···⃗v n ). Thus U is the matrix which has the ⃗u i as columns and V is defined as the matrix
which has the⃗v i as columns. Then
⎡
⃗u T ⎤
1.
U T AV =
⃗u T A[⃗v
⎢ ⎥ 1 ···⃗v n ]
⎣
ḳ. ⎦
j
⎡
=
⎢
⎣
⃗u T 1.
⃗u T ḳ.
⃗u T m
⎤
⃗u T m
[ σ1 ⃗u
⎥ 1 ··· σ k ⃗u k
⃗0 ··· ⃗0 ] [ ] σ 0
=
0 0
⎦