Numerical Methods Course Notes Version 0.1 (UCSD Math 174, Fall ...

46 CHAPTER 3. SOLVING LINEAR SYSTEMS
(e) Show that when 0 < α, this quantity is always smaller than 1.
(f) Prove that if A is positive definite, then there is an ω such that Richardson’s Iteration
with this ω will converge for any choice of x (0) .
(g) For which matrix do you expect faster convergence of Richardson’s Iteration: A 1 with
eigenvalues in [10, 20] or A 2 with eigenvalues in [1010, 1020]? Why?
(3.11) Implement Richardson’s Iteration to solve the system Ax = b. Your m-file should have
header line like:
function xk = richardsons(A,b,x0,w,k)
Your code should return x (k) based on the iteration
( )
x (j+1) = x (j) − ω Ax (j) − b .
Let w take the place of ω, and let x0 be the initial iterate x (0) . Test your code for A, b for
which you know the actual solution to the problem. (Hint: Start with A and the solution x
and generate b.) Test your code on the following matrices:
• Let A be the Hilbert Matrix. This is generated by the octave command A = hilb(10),
or whatever (smallish) integer. Try different values of ω, including ω = 1.
• Let A be a Toeplitz matrix of the form:
⎡
A =
⎢
⎣
−2 1 0 · · · 0
1 −2 1 · · · 0
0 1 −2 · · · 0
. . .
. .. .
0 0 0 · · · −2
These can be generated by the octave command A = toeplitz([-2 1 0 0 0 0 0 0]).
Try different values of ω, including ω = −1/2.
(3.12) Let A be a nonsingular n × n matrix. We wish to solve Ax = b. Let x (0) be some starting
vector, let D k be span { r (0) , Ar (0) , . . . , A k r (0)} , and let P k be the set of polynomials, p(x) of
degree k with p(0) = 1.
Consider the following iterative method: Let x (k+1) be the x that solves
min
x∈x (0) +D k
‖b − Ax‖ 2
.
Let r (k) = b − Ax (k) .
(a) Show that if x ∈ x (0) + D k , then b − Ax = p(A)r (0) for some p ∈ P k .
(b) Prove that, conversely, for any p ∈ P k there is some x ∈ x (0) + D k , such that b − Ax =
p(A)r (0) .
(c) Argue that ∥ ∥ ∥∥r (k+1)∥
∥∥p(A)r (0)
∥ = min ∥ .
2 p∈P k 2
(d) Prove that this iteration converges in at most n steps. (Hint: Argue for the existence
of a polynomial in P n that vanishes at all the eigenvalues of A. Use this polynomial to
show that ∥ ∥ r
(n) ∥ ∥
2
≤ 0.)
⎤
⎥
⎦