The SOR method for infinite systems of linear equations (III)

CARPATHIAN J. MATH.22 (2006), No. 1 - 2, 49 - 55The SOR method for infinite systems of linearequations (III)BÉLA FINTAABSTRACT. In [3], [4] we presented the extension of the Jacobi and Gauss-Seidel iterative numericalmethod from the case of finite linear systems to the case of infinite systems.The purpose of this paper is to extend the classical SOR (successiv over relaxation) method, knownfor finite linear systems, to infinite systems.Let x =⎛⎜⎝x 0x 1.x n..1. VECTOR NORMS⎞be a sequence of real numbers represented in the form of⎟⎠an infinite column vector, and we denote by s the real linear space of these sequences.Let us define l 1 = {x ∈ s | |x i | is convergent}. It is well known∞∑thatl 1 is a real linear subspace of s and for every x ∈ l 1 the formula ‖x‖ 1 =i=0∞∑|x i |defines a norm on l 1 . In this way (l 1 , ‖·‖ 1 ) is not only a normed linear space,but a Banach space, too. We will call it vector space, the elements vectors and theabove mentioned norm, vector norm [6], [8]. For this paragraph see also [2].i=02. MATRIX NORMSLet A =(a ij ) i,j∈N be an infinite matrix of real numbers and we denote by M{∞∑the real linear space of these infinite matrices. Let M 1 = A ∈ M | sup |a ij |j∈Ni=0is finite}. Then M 1 is a real linear subspace of M and for every A ∈ M 1 the∞∑formula ‖A‖ 1 =sup |a ij | defines a norm on M 1 called column norm. In thisj∈Ni=0way (M 1 , ‖·‖ 1 ) becomes not only a real linear normed space, but a Banach space,too.Received: 15.02.2006; In revised form: 10.10.2006; Accepted: 01.11.20062000 Mathematics Subject Classification: 65F10, 15A60.Key words and phrases: Space l 1 , infinite systems of linear equations, SOR iterative method, Gauss-Seidel’s iterative method, Banach fixed point theorem.49

50 Béla FintaCorollary 2.1. If for the matrix A =(a ij ) i,j∈N we have a ij =0for i>nand j>n,n ∈ N, then from the above we obtain the results in the finite dimensional space R n [1],[5].This space will be called matrix space and the above mentioned norm, matrixnorm. For this paragraph see also [2], [6], [8].3. THE COMPATIBILITY OF THE VECTOR AND MATRIX NORMSLet x ∈ s be a sequence of real numbers, and A =(a ij ) i,j∈N ∈ M an infinitematrix of real numbers.Definition 3.1. We will define the product A · x if for every i ∈ N the series∞∑a ij x j is convergent. In this case the resulting vector y = A · x is a columnj=0⎛vector with components y =⎜⎝∞∑⎞a 0j x jj=0∞∑a 1j x jj=0...∞∑a ij x j⎟j=0 ⎠.Theorem 3.1. The vector norm ‖·‖ 1 defined on l 1 is compatible with the matrix norm‖·‖ 1 defined on M 1 , i.e. ‖Ax‖ 1 ≤‖A‖ 1 ·‖x‖ 1 for every x ∈ l 1 and every A ∈ M 1 .Proof. We have:‖A · x‖ 1 ==≤∣ ∣∣∣∣∣ ∞∑∞∑∞∑ ∞∑a ij x j ≤ |a ij |·|x j | =∣j=0i=0 j=0∞∑ ∞∑∞∑ ∞∑|a ij |·|x j | = |x j |· |a ij |≤i=0j=0 i=0∞∑j=0|x j |·supj∈N∞∑i=0j=0|a ij | =supj∈Ni=0∞∑ ∞∑|a ij |· |x j | = ‖A‖ 1 ·‖x‖ 1 .i=0j=0□Corollary 3.2. If for the matrix A =(a ij ) i,j∈N we have a ij =0for i>nand j>n,n ∈ N, then from Theorem 3.1 we reobtain the results in the finite dimensional space R n[1], [5].For this paragraph see also [2], [6], [8].

The SOR method for infinite systems of linear equations (III) 514. THE MATRIX NORM SUBORDINATED TO A GIVEN VECTOR NORMFor every x ∈ l 1 and A ∈ M 1 we have ‖Ax‖ 1 ≤‖A‖ 1 ·‖x‖ 1 according toTheorem 3.1. If x ≠ θ l 1 (the null element of the vector space l 1 ), then ‖Ax‖ 1≤{ }‖x‖ 1‖Ax‖1‖A‖ 1 and we can define sup | x ∈ l 1 \{θ l 1} .‖x‖ 1It is known that this formula defines a matrix norm on M 1 , which we call thematrix norm subordinated { to the vector norm } ‖·‖ 1 defined on l 1 and we denoteit by ‖A‖ ∗ ‖Ax‖11 =sup | x ∈ l 1 \{θ l 1} . It is immediately that ‖A‖ ∗ 1 ≤‖A‖ 1 ,‖x‖ 1for every A ∈ M 1 . Actually, we haveTheorem 4.2. ‖A‖ ∗ 1 = ‖A‖ 1.∞∑Proof. We must prove that ‖A‖ ∗ 1 ≥‖A‖ 1 . From ‖A‖ 1 =sup |a ij | we obtain: forj∈Ni=0∞∑every ε>0 there exists j 0 ∈ N such that |a ij0 |≥‖A‖ 1 − ε. Let us choose thevector x ∈ l 1 such that all the components of x are zero except for the componentj 0 . So∣ ∣∣∣∣∣ ∞∑∞∑∞∑‖A · x‖ 1 =a ij x j = |a ij0 x j0 | =i=0 ∣j=0i=0(∞∑∞)∑= |a ij0 |·|x j0 | = |a ij0 | ·|x j0 |≥Consequentlyi=0i=0i=0≥ (‖A‖ 1 − ε) ·|x j0 | =(‖A‖ 1 − ε) ·‖x‖ 1 .‖Ax‖ 1≥‖A‖ 1 − ε,‖x‖ 1i.e.{ }‖A‖ ∗ ‖Ax‖11 =sup | x ∈ l 1 \{θ l 1} ≥‖A‖ 1 − ε,‖x‖ 1for every ε>0. This means that ‖A‖ ∗ 1 ≥‖A‖ 1.□Corollary 4.3. If for the matrix A =(a ij ) i,j∈N we have a ij =0for i>nand j>n,n ∈ N, then from Theorem 4.2 we reobtain the results in the finite dimensional space R n[1], [5].For this paragraph see also [2], [6], [8].The above presented vector and matrix spaces will be used to extend the iterativeJacobi’s and Gauss-Seidel’s methods, from finite linear systems to the caseof infinite systems. In this way we can study the linear stationary processes withinfinite but countable number of parameters.

52 Béla Finta5. THE SOR METHOD FOR INFINITE SYSTEMS OF LINEAR EQUATIONSLet us consider the infinite system of linear equations Ax = b, where A ∈ Mand x, b ∈ s.Definition 5.2. For a given A ∈ M and b ∈ s we will say that x ∗ ∈ s is a solutionof the infinite system of linear equations Ax = b if we have Ax ∗ = b.∞∑∞∑This means that the series a ij x ∗ j is convergent and we have a ij x ∗ j = b i ,j=0for every i ∈ N.Let us suppose that a ii ≠0for every i ∈ N and let us consider the constantsω i ∈ R \{0} for every i ∈ N. The initial system of linear equations Ax = b isequivalent to the following iterative system of linear equations:⎧∑∞ a 0j b 0x 0 =(1− ω 0 )x 0 − ω 0 x j + ω 0a 00⎪⎨⎪⎩.j=1x 1 = −ω 1a 10a 11x 0 +(1− ω 1 )x 1 − ω 1a 00∑∞j=2a 1ja 11x j + ω 1b 1x 2 = −ω 2a 20a 22x 0 − ω 2a 21a 22x 1 +(1− ω 2 )x 2 − ω 2.∑i−1a ijx i = −ω i x j +(1− ω i )x i − ω ia iij=0∞ ∑j=i+1a 11∑∞j=3a ija iix j + ω ib ia iij=0a 2ja 22x j + ω 2b 2a 22Using this system of linear equations, let us choose x 0 ∈ s and we generate thesequence (x k ) k∈N ⊂ s by the following iterative formula:⎧∞∑x k+10 =(1− ω 0 )x k 0 − ω a 0j0 x k ja + ω b 0000(5.1)⎪⎨⎪⎩.j=1a 00x k+1 a 10∑∞1 = −ω 1 x k+10 +(1− ω 1 )x k a 1j1 − ω 1 x k b 1j + ω 1a 11 a 11 a 11x k+12 = −ω 2a 20a 22x k+10 −ω 2a 21a 22tx k+1.x k+1i∑i−1a ij= −ω i x k+1ja iij=0j=21 +(1−ω 2 )x k 2 −ω 2 ∞ ∑+(1− ω i )x k i − ω i∞∑j=i+1a ijj=3a iix k j + ω ia 2ja 22x k j +ω 2b 2a 22Consequently, starting from the vector x k , we generate the vector x k+1 by therecursion formula x k+1 = B ω · x k + c.b ia ii

The SOR method for infinite systems of linear equations (III) 53Definition 5.3. The matrix A =(a ij ) i,j∈N is l 1 diagonal dominated if there existsthe positive real number λ > 0 such that for every j ∈ N we have∞∑λ|a jj | > |a ij |.i=0i≠jIt is immediately that A is l 1 diagonal dominated if and only if supj∈Na finite real number.Let us denote by λ := supj∈N∞∑∣ a ij ∣∣∣∣ isa jji=0i≠j∣ ∞∑a ij ∣∣∣∣ ,ω ∗ =sup{|ω i |/i ∈ N} ∈R and ω ∗∗ =a jji=0i≠jsup{|1 − ω i |/i∈ N} ∈R. Let us suppose ω ∗ · λ

54 Béla FintaConsequently:‖y‖ 1 ≤ ω ∗ λ‖x‖ 1 + ω ∗ λ‖y‖ 1 + ω ∗∗ ‖x‖ 1 ,which is equivalent to‖y‖ 1≤ ω∗ λ + ω ∗∗‖x‖ 1 1 − ω ∗ λ .This means that‖B ω x‖ 1 ‖y‖ 1‖B ω ‖ 1 = sup = sup ≤ ω∗ λ + ω ∗∗x≠θ l‖x‖1 1 x≠θ l‖x‖1 1 1 − ω ∗ λ < 1.Now we can apply the Banach fixed point theorem for the iteration map Φ:l 1 → l 1 , Φ(x) =B ω x + c. Indeed, Φ is a contraction, because‖Φ(x) − Φ(y)‖ 1 = ‖(B ω x + c) − (B ω y + c)‖ 1 = ‖B ω (x − y)‖ 1 ≤‖B ω ‖ 1 ‖x − y‖ 1 .This means that the sequence (x k ) k∈N is convergent in l 1 for every x 0 ∈ l 1 andits limit point x ∗ ∈ l 1 is the unique fixed point of Φ in l 1 , i.e. Φ(x ∗ )=x ∗ . SoB ω x ∗ + c = x ∗ , which is equivalent to Ax ∗ = b.□Corollary 5.4. If for the matrix A =(a ij ) i,j∈N we have a ij =0when i>n,j>nand b i =0for i>n,n∈ N, then we reobtain the linear system with finite number ofequations and finite number of unknowns. In this way from Theorem 5.3 we obtain theclassical SOR iterative numerical method to solve finite systems of linear equations [7].In the following we consider the particular case when ω i = ω, for every i ∈ N.So, from (5.1) we can deduce: let us choose x 0 ∈ s and we generate the sequence(x k ) k∈N ⊂ s by the following iterative formula:(5.2)⎧⎪⎨⎪⎩.∞ x k+10 =(1− ω)x k 0 − ω ∑ a 0jx k j + ω b 0a 00j=1x k+11 = −ω a 10a 11x k+10 +(1− ω)x k 1 − ωx k+12 = −ω a 20x k+10 − ω a 21x k+1a 22 a 22..x k+1i∑i−1= −ωj=0a 00∞∑a 1jx k j + ω b 1aj=2 11 a 11∞1 +(1− ω)x k 2 − ω ∑a ija iix k+1j +(1− ω)x k i − ω ∞ ∑In this case from Theorem 5.3 we obtain:j=i+1j=3a ija iix k j + ω b ia iia 2ja 22x k j + ω b 2a 22|ω|λ + |1 − ω|Corollary 5.5. If < 1, (|ω|λ < 1) then the corresponding iterative1 −|ω|λsequence (x k ) k∈N given by (5.2) is convergent in l 1 for every x 0 ∈ l 1 . The limit pointx ∗ ∈ l 1 is the unique solution of the linear system Ax = b.

The SOR method for infinite systems of linear equations (III) 55In the following we consider the particular case, when ω i = ω =1for everyi ∈ N. So from (5.2) we can deduce: let us choose x 0 ∈ s and we generate thesequence (x k ) k∈N ⊂ s by the following iterative formula:(5.3)⎧⎪⎨x k+10 = −∞∑j=1a 0ja 00x k j + b 0x k+11 = − a 10a 11x k+10 −a 00∞∑j=2x k+12 = − a 20a 22x k+10 − a 21a 22x k+1.x k+1i∑i−1a ij= − x k+1ja iij=0−a 1ja 11x k j + b 11 −∞∑j=i+1a 11∞∑j=3a 2ja 22x k j + b 2a 22a ija iix k j + b ia ii⎪⎩.From Theorem 5.3 and Corollary 5.5 we obtain the author’s result, the Gauss-Seidel’s iterative method for infinite systems of linear equations [4]:Corollary 5.6. If λ< 1 2 , then the corresponding iterative sequence (xk ) k∈N given by(5.3) is convergent in l 1 for every x 0 ∈ l 1 . The limit point x ∗ ∈ l 1 is the unique solutionof the linear system Ax = b.We can obtain similar results if we replace the space l 1 by the space l ∞ or l p ,for p ∈ (1, +∞).We mention that all these results are valid in the complex case, too.REFERENCES[1] Finta, B., A Remark about Vector and Operator Norms, Mathematical Magazine Octogon, Braşov, 6,No. 1, April, 1998, 58-60[2] Finta, B., A Remark about Vector and Matrix Norms (II), Buletin Ştiinţific, XV-XVI, Universitatea”Petru Maior”, Tg. Mureş, 2002-2003, 113-120[3] Finta, B., Jacobi’s Theorem for Infinite System of Linear Equations, Profesor Gheorghe Fărcaşlavârsta de70 de ani, Volum omagial, Editura Universităţii ”Petru Maior” Tg. Mureş, 2004, 103-108[4] Finta, B., Gauss-Seidel’s Theorem for Infinite System of Linear Equations (II), International Conference”European Integration Between Tradition and Modernity”, IETM First Edition, ”Petru Maior”University of Tg. Mures, 22-24 September 2005, Romania, 669-677[5] Horn, R. A., Johnson, C. R., Matrix Analysis, Cambridge University Press, Cambridge, 1990[6] Kantorovitch, L., Akilov, G., Analyse fonctionnelle, Éditions MIR, Moscou, 1981[7] Plato, R., Concise Numerical Mathematics, Graduate Studies in Mathematics, 57, AMS, 2003[8] Szidarovszky, F., Yakowitz, S., Principles and Procedures of Numerical Analysis, Plenum Press,New York and London, 1978DEPARTMENT OF MATHEMATICS”PETRU MAIOR”UNIVERSITY TG. MUREŞNICOLAE IORGA 1, 540088 TG. MUREŞ, ROMANIAE-mail address: fintab@upm.ro