Numerical Methods Contents - SAM

Goal:
Euclidean distance of y ∈ R d from the plane: dist(H,y) = |c + n T y| . (6.2.7)
given the points y 1 ,...,y m , m > d, find H ↔ {c ∈ R,n ∈ R d , ‖n‖ 2 = 1}, such that
m∑
m∑
dist(H,y j ) 2 = |c + n T y j | 2 → min . (6.2.8)
j=1
Note: (6.2.8) ≠ linear least squares problem due to constraint ‖n‖ 2 = 1.
⎛
⎞⎛
⎞
1 y 1,1 · · · y 1,d c
(6.2.8) ⇔
⎜1 y 2,1 · · · y 2,d
⎟⎜n 1
⎟
⎝
. . . ⎠⎝
. ⎠
→ min under constraint ‖n‖ 2 = 1 .
1 y m,1 · · · y m,d n
∥} {{ } d ∥
=:A
2
Step ➊: QR-decomposition (→ Section 2.8)
⎛
⎞
r 11 r 12 · · · · · · r 1,d+1
⎛
⎞
1 y 1,1 · · · y 1,d
0 r 22 · · · · · · r 2,d+1
A := ⎜1 y 2,1 · · · y 2,d
⎟
⎝.
. . ⎠ = QR , R := . ... .
0 r d+1,d+1
∈ R m,d+1 .
1 y m,1 · · · y m,d
⎜ 0 · · · · · · 0
⎟
⎝ .
. ⎠
0 · · · · · · 0
j=1
⎛
⎞
r 11 r 12 · · · · · · r 1,d+1 ⎛ ⎞
0 r 22 · · · · · · r 2,d+1
c
. . .. .
n 1
‖Ax‖ 2 → min ⇔ ‖Rx‖ 2 =
0 r d+1,d+1
⎜ .
⎟
→ min . (6.2.9)
⎜ 0 · · · · · · 0
⎝
⎟
. ⎠
⎝
.
. ⎠ n d ∥ 0 · · · · · · 0 ∥ 2
Step ➋
Note that necessarily (why?)
Ôº¾½ º¾
Note: Since r 11 = ∥ ∥(A) :,1
∥ ∥2 = √ d + 1 ≠ 0 ⇒ c = −r −1
11
MATLAB-function:
For A ∈ K m,n find
n ∈ R d , c ∈ R n−d
such that
( c ∥∥∥2
∥ n)∥ A → min
with the constraint:
‖n‖ 2 = 1 .
6.3 Total Least Squares
d∑
r 1,j+1 n j .
j=1
Code 6.2.5: (Generalized) distance fitting a hyperplane
1 function [ c , n ] = clsq (A, dim ) ;
2 [m, p ] = size (A) ;
3 i f p < dim+1 , error ( ’ not enough unknowns ’ ) ; end ;
4 i f m < dim , error ( ’ not enough equations ’ ) ; end ;
5 m = min (m, p ) ;
6 R = t r i u ( qr (A) ) ;
7 [U, S, V ] = svd (R( p−dim +1:m, p−dim +1:p ) ) ;
8 n = V ( : , dim ) ;
9 c = −R( 1 : p−dim , 1 : p−dim ) \R( 1 : p−dim , p−dim +1:p ) ∗n ;
Given: overdetermined linear system of equations Ax = b, A ∈ R m,n , b ∈ R m , m ≥ n.
Known: LSE solvable ⇔ b ∈ Im(A), if A, b were not perturbed,
Sought:
but A, b are perturbed (measurement errors).
Solvable overdetermined system of equations Âx = ̂b, Â ∈ Rm,n , ̂b ∈ R m , “nearest”
to Ax = b.
✸
Ôº¾¿ º¿
This insight converts (6.2.9) to
c · r 11 + n 1 · r 12 + · · · + r 1,d+1 · n d = 0 .
⎛
⎞⎛
⎞
r 22 r 23 · · · · · · r 2,d+1 n 1
⎜ 0 r 33 · · · · · · r 3,d+1
⎟⎜
.
⎟
⎝
. ... . ⎠⎝
. ⎠
→ min , ‖n‖ 2 = 1 . (6.2.10)
∥ 0 r d+1,d+1 n d
∥ 2
Ôº¾¾ º¾
(6.2.10) = problem of type (5.5.5), minimization on the Euclidean sphere.
➣ Solve (6.2.10) using SVD !
☞ least squares problem “turned upside down”: now we are allowed to tamper with system matrix
and right hand side vector!
Total least squares problem:
Given: A ∈ R m,n , m ≥ n, rank(A) = n, b ∈ R m ,
find:
 ∈ R m,n , ̂b ∈ R m with
]
∥
∥[A b] −
[Â ̂b
} {{ }
∥ → min , ̂b ∈ Im( Â) .
} {{ } F
=:C
=:Ĉ
Ôº¾ º¿
(6.3.1)