Numerical Methods Contents - SAM

∥ ∥∥C ̂b ∈ Im(Â) ⇒ rank(Ĉ) = n (6.3.1) ⇒ min − Ĉ∥ .
rank(Ĉ)=n F
Thm. 5.5.7 ➤ use the SVD decomposition of C to construct Ĉ:
Algorithm (based on block-LU-decomposition):
(
A H A C H ) ( )( )
R H 0 R G H
=
C 0 G −S H 0 S
,
R,S ∈ R n,n upper triangular matrix,
G ∈ R p,n .
Ĉ =
n+1
C = UΣV H ∑
= σ j (U) :,j (V) H :,j
j=1
n∑
σ j (U) :,j (V) H :,j ⇒ Ĉ(V) :,n+1 = 0 .
j=1
If (V) n+1,n+1 ≠ 0, then
Âx = ̂b with
x = (v) −1
n+1,n+1 (V)·,n+1 .
Code 6.3.2: Total least squares via SVD
1 function x = l s q t o t a l (A, b ) ;
2 [m, n ]= size (A) ;
3 [U, Sigma , V ] = svd ( [ A, b ] ) ;
4 s = V( n+1 ,n+1) ;
5 i f s == 0 ,
6 error ( ’No s o l u t i o n ’ )
7 end
8 x = −V( 1 : n , n+1) / s ;
Caution
R from R H R = A H A → Cholesky decomposition → Sect. 2.7,
G from R H G H = C H → n forward substitution → Sect. 2.2,
S from S H S = GG H → Cholesky decomposition → Sect. 2.7.
Sect. 6.1: the computation of A H A can be expensive and problematic!
(remedy through introduction of a new unknown r = Ax − b , cf. (6.1.3))
⎛ ⎞⎛
−I A 0
⎝A H 0 C H ⎠⎝ r ⎞ ⎛
x ⎠ = ⎝ b ⎞
0⎠ . (6.4.2)
0 C 0 m d
Solution via SVD:
6.4 Constrained Least Squares
Given: A ∈ R m,n , m ≥ n, rank(A) = n, b ∈ R m ,
C ∈ R p,n , p < n, rank(C) = p, d ∈ R p
Find: x ∈ R n with ‖Ax − b‖ 2 → min , Cx = d .
Linear constraint
(6.4.1)
Ôº¾ º
➀ Compute orthonormal basis of Ker(C) using SVD (→ Section 6.2):
[ ]
V H
C = U [Σ 0] 1
V2
H Ker(C) = Im(V 2 ) .
and the particular solution
Ôº¾ º
, U ∈ R p,p , Σ ∈ R p,p , V 1 ∈ R n,p , V 2 ∈ R n,n−p
x 0 := V 1 Σ −1 U H d .
Representation of the solution x of (6.4.1): x = x 0 + V 2 y, y ∈ R n−p .
➁ Insert this representation in (6.4.1) ➤ standard linear least squares
Solution via normal equations
‖A(x 0 + V 2 y) − b‖ 2 → min ⇔ ‖AV 2 y − (b − Ax 0 )‖ → min .
Idea: coupling the constraint using the Lagrange multiplier m ∈ R p
x = argmin
x∈R n
max L(x,m) , L(x,m) := 1 m∈R p 2 ‖Ax − b‖2 + m H (Cx − d) .
Necessary (and sufficient) condition for the solution (→ Section 6.1)
(
A H A C H
C 0
∂L
∂x (x,m) = AH (Ax − b) + C H m ! = 0 ,
) (
x
m
)
=
(
A H )
b
d
∂L
∂m (x,m) = Cx − d ! = 0.
Extended normal equations (saddle point problem)
Ôº¾ º
Exercise 6.4.1. Given a regular tridiagonal matrix T ∈ R n,n , develop an algorithm for solving the
linear least squares problem
where
x ∗ = argmin
x∈R n ‖Ax − b‖ 2 ,
⎛ ⎞
A = ⎝ T−1
. ⎠ ∈ R pn,n .
T −1
Ôº¾ º