v2007.09.17 - Convex Optimization

A.6. SINGULAR VALUE DECOMPOSITION, SVD 507A.6 Singular value decomposition, SVDA.6.1Compact SVD[110,2.5.4] For any A∈ R m×nq T1A = UΣQ T = [ u 1 · · · u η ] Σ⎣.⎡⎤∑⎦ = η σ i u i qiTqηT i=1(1350)U ∈ R m×η , Σ ∈ R η×η , Q ∈ R n×ηwhere U and Q are always skinny-or-square each having orthonormalcolumns, and whereη ∆ = min{m , n} (1351)Square matrix Σ is diagonal (A.1.1)δ 2 (Σ) = Σ ∈ R η×η (1352)holding the singular values {σ i ∈ R} of A which are always arranged innonincreasing order by convention and are related to eigenvalues by A.14⎧√ ⎨ λ(AT A)σ(A) i = σ(A T i = √ (√ )λ(AA T ) i = λ AT A)= λ(√AAT> 0, 1 ≤ i ≤ ρ) i =ii⎩0, ρ < i ≤ η(1353)of which the last η −ρ are 0 , A.15 whereρ ∆ = rankA = rank Σ (1354)A point sometimes lost: Any real matrix may be decomposed in terms ofits real singular values σ(A) ∈ R η and real matrices U and Q as in (1350),where [110,2.5.3]R{u i |σ i ≠0} = R(A)R{u i |σ i =0} ⊆ N(A T )R{q i |σ i ≠0} = R(A T (1355))R{q i |σ i =0} ⊆ N(A)A.14 When A is normal,√σ(A) = |λ(A)|. [301,8.1])A.15 For η = n , σ(A) = λ(A T A) = λ(√AT A where λ denotes eigenvalues.√)For η = m , σ(A) = λ(AA T ) = λ(√AAT.