Masters Thesis - TU Delft

Chapter 3Conjugate Gradient Method3.1 IntroductionOne of our main problems is to solve a linear system:Ax = b, (3.1)where A ∈ R n×n is called a coefficient matrix, b ∈ R n a right-hand side vector, and wheren ∈ N. Keeping in mind the fact, that we are dealing with matrices from the discretization, wewill assume that A is symmetric and semi-positive definite.There are many ways to solve this problem. In this chapter we will focus on a well-knowniterative method, called Conjugate Gradient or, in short-hand notation CG, developed by E.Stiefel and by M.R Hestenes[3], which allows us to compute the solution of the above mentionedsystem of equations. The success of this algorithm lies in it’s simplicity. However, to describethe Conjugate Gradient method precisely, we have to know exactly what is a basic iterativemethod.3.2 Basic Iterative MethodsBasic iterative solution methods are used to generate a sequence (x i ,i = 0,1,...) which maybe finite or not, consisting of the approximations of the exact solution x. The compute thissequence, the following recursive formula is used,x i+1 = x i +M −1 (b−Ax i ) (3.2)We can substitute r i = b − Ax i , to which we will from now on refer as the i-th residual,which is used to measure the difference of the i-th approximation and the exact solution andrewrite the above equation once again in a more pleasant way,If we now write the first steps of this iteration process,x i+1 = x i +M −1 r i (3.3)11