Numerical Methods Contents - SAM

0 2 4 6 8 10 12 14 16
value of ∥ ∥ F(x
(k) )
∥ ∥
2
18
16
14
2
12
10
8
6
4
2
0
0 2 4 6 8 10 12 14 16 10−16
Damped Newton method
Convergence behaviour of the Newton method:
10 2
10 0
10 −2
10 −4
10 −6
10 −8
10 −10
10 −12
10 −14
norm of grad Φ(x (k) )
value of ∥ ∥ F(x
(k) )
∥ ∥
2
18
16
14
2
12
10
8
6
4
2
0
0 2 4 6 8 10 12 14 16 10−16
Not−damped Newton method
10 4
10 2
10 0
10 −2
10 −4
10 −6
10 −8
10 −10
10 −12
10 −14
norm of grad Φ(x (k) )
Idea: damping of the Gauss-Newton correction in (6.5.7) using a penalty term
∥
instead of ∥F(x (k) ) + DF(x (k) )s∥ 2
∥
minimize ∥F(x (k) ) + DF(x (k) )s∥ 2 + λ ‖s‖ 2 2 .
λ > 0 ˆ= penalty parameter (how to choose it ? → heuristic)
⎧
∥
10 , if ∥F(x ⎪⎨
(k) ) ∥ ≥ 10 ,
2 ∥
λ = γ ∥F(x (k) ∥
) ∥ , γ := 1 , if 1 < ∥F(x (k) ) ∥ < 10 ,
2 2
⎪⎩ ∥
0.01 , if ∥F(x (k) ) ∥ ≤ 1 . 2
Modified (regularized) equation for the corrector s:
(
)
DF(x (k) ) T DF(x (k) ) + λI s = −DF(x (k) )F(x (k) ) . (6.5.8)
initial value (1.8, 1.8, 0.1) T (red curve) ➤ Newton method caught in local minimum,
initial value (1.5, 1.5, 0.1) T (cyan curve) ➤ fast (locally quadratic) convergence.
Ôº¿ º
Ôº¿ º
0.9
10 0
0.8
10 −2
Gauss-Newton method:
initial value (1.8, 1.8, 0.1) T (red curve),
initial value (1.5, 1.5, 0.1) T (cyan curve),
convergence in both cases.
Notice:
linear convergence.
value of ∥ ∥ F(x
(k) )
∥ ∥
2
0.7
2
0.6
0.5
0.4
0.3
0.2
0.1
10 −4
10 −6
10 −8
10 −10
10 −12
10 −14
norm of the corrector
7 Filtering Algorithms
Perspective of signal processing:
vector x ∈ R n ↔ finite discrete (= sampled) signal.
0
0 2 4 6 8 10 12 14 16 10−16
Gauss−Newton method
✸
X = X(t) ˆ= time-continuous signal, 0 ≤ t ≤ T ,
“sampling”: x j = X(j∆t) , j = 0, ...,n − 1,
n ∈ N, n∆t ≤ T .
∆t > 0 ˆ= time between samples.
6.5.3 Trust region method (Levenberg-Marquardt method)
Sampled values arranged in a vector x =
(x
As in the case of Newton’s method for non-linear systems of equations, see Sect. 3.4.4: often overshooting
of Gauss-Newton corrections occurs.
Ôº¿ º
0 , ...,x n−1 ) T ∈ R n .
Note: vector indices 0, ...,n − 1 !
(“C-style indexing”).
Remedy as in the case of Newton’s method: damping.
X = X(t)
x 0
x
x n−2 1
x
x n−1 2
t 0 t 1 t 2 t n−2 t n−1 Fig. time 90
Ôº¼ º½