Introduction to Krylov subspace methods - IMAGe

For a SPD matrix A, considerg(x) := 1 2 〈Ax, x〉 − 〈b, x〉 where 〈u, v〉 := m∑Minimization i=1g(x) := 1 m2 〈Ax, x〉 − 〈b, x〉 where 〈u, v〉 := ∑u i v i . i=1u i v i .ThenThen(∇g(x)) k = dgdmdx k= 1 d ∑ma ij x i x j −ddx kdxdx i,j=1kk 2 dx k= 1 i,j=1m∑a ij δ ik x j + 1 m∑a ij δ jk x i −2m∑2m∑(∇g(x)) k = dg= 1 2= 1 2i,jai,jij δ ik x j + 1 ∑ 2= 1 ∑i,j a kj x j + 1 22∑j∴ j (∇g) k =ia ik xi,ji − b km∑a kj ix j − b kj=1∇g = Ax − b = 0 ⇐⇒ Ax ∗ = b,Condition at the minimum:m∑i=1b i x ii=1m∑b i δ ikai=1 ij δ jk x i −x ∗ : solution of Ax = bIt means that the solution of Ax = b is also the minimum of g.j=1Now, what’s the best and simplest method to find a minimum?Starting with a ”guess” x 0 ∈ R m and a non-zero vector p ∈ R m with step length αdetermined in a way to minimizex 1 = x 0 + αpa kj x j + 1 2∴ (∇g) k =∇g = Ax − b = 0 ⇐⇒ Ax ∗ = b,a ij x i x j −∑a ik x i − b km∑a kj x j − b kddx km ∑b i x im∑b i δ iki=1x ∗ : solution of Ax = beans that the solution of Ax = b is also the minimum of g.11