An Introduction to the Conjugate Gradient Method Without the ...

¡¡26 Jonathan Richard Shewchuk422-4 -2 2 4 61-2-4-6Figure 25: The method of Conjugate Directions using the axial unit vectors, also known as Gaussianelimination.are required to generate the full set. In fact, if the search vectors are constructed by conjugation of the axialunit vectors, Conjugate Directions becomes equivalent to performing Gaussian elimination (see Figure 25).As a result, the method of Conjugate Directions enjoyed little use until the discovery of CG — which is amethod of Conjugate Directions — cured these disadvantages.An important key to understanding the method of Conjugate Directions (and also CG) is to noticethat Figure 25 is just a stretched copy of Figure 21! Remember that when one is performing the methodof Conjugate Directions (including CG), one is simultaneously performing the method of OrthogonalDirections in a stretched (scaled) space.7.3. Optimality of the Error TermConjugate Directions has an interesting property: it finds at every step the best solution within the boundsof where it’s been allowed to explore. Where has it been allowed to explore? ¥ Let be the -dimensional1¡ 1¡¢subspace ; the ¢¡ value is chosen ¢ 0¡ from . What do I mean by “bestspan¡ 0¡solution”? I mean that Conjugate Directions chooses ¢ 0¡the value from ¢¡§that minimizes (seeFigure 26). In fact, some authors derive CG by trying to ¢¡§minimize ¢ within .§ §0¡In the same way that the error term can be expressed as a linear combination of search directions