Abstract

APPENDIX D. GMRES AND ARNOLDI ITERATIVE METHODS 152
be the j-th Krylov iterate. So a j solves the least-squares problem
min
a∈a 0 r2 = Da −
+Kj
b2. Let {pi} j
i=1 form an orthonormal basis of the Krylov space Kj. Then every vector
v ∈ Kj can be written as
v =
j
γipi.
i=1
If we take the vectors {pi} j
i=1 and form a matrix Pj ∈ R M×j where the columns of Pj
are the vectors {pi} j
i=1 , then every vector v ∈ Kj can be written as
v = Pjγ
for some γ ∈ R j . Therefore, the j-th Krylov iteration satifies
min
a∈a 0 Da −
+Kj
b2 =min
γ∈R j D(a0 + Pjγ) −b2 =min
γ∈R j DPjγ − r 0 2.
This means to compute the j-th Krylov iteration, we can solve an j-dimensional linear
least squares problem for the coefficients γ. This j-dimensional linear least-squares
problem can be solved with a QR factorization or an SVD [28], but to do so we need
to compute the matrix Pj. Now,thecolumnsofPj can be computed by applying the
Gram-Schmidt procedure to the set {r 0 ,Dr 0 ,D 2 r 0 ,...,D j−1 r 0 }. This computational
algorithm for determining Pj is known as the Arnoldi procedure [19]. The Arnoldi