Numerical Methods Contents - SAM

4.2.1 Krylov spaces
Definition 4.2.3 (Krylov space).
For A ∈ R n,n , z ∈ R n , z ≠ 0, the l-th Krylov space is defined as
{
}
K l (A,z) := Span z,Az, . ..,A l−1 z .
Assume:
A-orthogonal basis {p 1 ,...,p n } of R n available, such that
Span {p 1 ,...,p l } = K l (A,r) .
(Efficient) successive computation of x (l) becomes possible
(LSE (4.2.3) becomes diagonal !)
Equivalent definition: K l (A,z) = {p(A)z: p polynomial of degree ≤ l}
✬
Lemma 4.2.4. The subspaces U k ⊂ R n , k ≥ 1, defined by (4.2.1) and (4.2.2) satisfy
}
U k = Span
{r 0 ,Ar 0 , ...,A k−1 r 0 = K k (A,r 0 ) ,
where r 0 = b − Ax (0) is the initial residual.
✫
Proof. (by induction) Obviously AK k (A,r 0 ) ⊂ K k+1 (A,r 0 ). In addition
r k = b − A(x (0) + z) for some z ∈ U k ⇒ r k = }{{}
r 0 − }{{} Az
∈K k+1 (A,r 0 ) ∈K k+1 (A,r 0 )
✩
✪
º¾
.
Ôº¿¿
Input: : initial guess x (0) ∈ R n
Given: : A-orthogonal bases {p 1 , . ..,p l } of K l (A,r 0 ), l = 1,...,n
Output: : approximate solution x (l) ∈ R n of Ax = b
Task:
r 0 := b − Ax (0) ;
for j = 1 to l do { x (j) := x (j−1) + pT j r 0
p T j Ap p j }
j
(4.2.4)
Efficient computation of A-orthogonal vectors {p 1 ,...,p l } spanning K l (A,r 0 ) during the
iteration.
Ôº¿ º¾
Since U k+1 = Span {U k ,r k }, we obtain U k+1 ⊂ K k+1 (A,r 0 ). Dimensional considerations based
on Lemma 4.2.1 finish the proof.
✷
4.2.2 Implementation of CG
Lemma 4.2.1 implies orthogonality p j ⊥ r m := b − Ax (m) , 1 ≤ j ≤ m ≤ l
p T j (b − Ax(m) ) = p T j
(
b} −{{ Ax (0) } − ∑ m
k=1
=r 0
p T k r )
0
p T k Ap Ap k = 0 . (4.2.5)
k
(4.2.5) ⇒ Idea: Gram-Schmidt orthogonalization, of residuals r j := b − Ax (j)
w.r.t. A-inner product:
Assume:
basis {p 1 ,...,p l }, l = 1, ...,n, of K l (A,r) available
(4.2.1) ➣ set x (l) = x (0) + γ 1 p 1 + · · · + γ l p k .
j∑
p 1 := r 0 , p j+1 := (b − Ax (j) p T ) − k Ar j
} {{ } p
=:r T j k=1 k Ap p k , j = 1,...,l − 1 . (4.2.6)
k
For ψ(γ 1 , ...,γ l ) := J(x (0) + γ 1 p 1 + · · · + γ l p l ) holds
(4.2.1) ⇔ ∂ψ
∂γ j
= 0 , j = 1,...,l .
This leads to a linear system of equations by which the coefficients γ j can be computed:
⎛
⎝ pT 1 Ap 1 · · · p T 1 Ap ⎞⎛
l
.
. ⎠⎝ γ ⎞ ⎛
1
. ⎠ = ⎝ pT 1 r
⎞
. ⎠ . (4.2.3)
p T l Ap 1 · · · p T l Ap l γ l p T l r
Ôº¿ º¾
Great simplification, if {p 1 ,...,p l } A-orthogonal basis of K l (A,r): p T j Ap i = 0 for i ≠ j.
✬
Lemma 4.2.5 (Bases for Krylov spaces in CG).
If they do not vanish, the vectors p j , 1 ≤ j ≤ l, and r j := b − Ax (j) , 0 ≤ j ≤ l, from (4.2.4),
(4.2.6) satisfy
(i) {p 1 , ...,p j } is A-orthogonal basis von K j (A,r 0 ),
(ii) {r 0 ,...,r j−1 } is orthogonal basis of K j (A,r 0 ), cf. Cor. 4.2.2
✫
✩
Ôº¿ º¾
✪