- Text
- Matrix,
- Algorithm,
- Vector,
- Matrices,
- Method,
- Sparse,
- Pivoting,
- Projection,
- Linear,
- Vectors,
- Meyer

A DIRECT PROJECTION METHOD FOR SPARSE ... - Carl Meyer

1160 m. benzi and c. d. meyerdenotes the matrix containing the first k rows of A, and if N k = N(A k ) denotesthe null space of A k , thenN 1 ⊃N 2 ⊃···⊃N n−1 ⊃N n = {0}.In order to solve the system of equations, we begin by constructing a set of null vectorsN = {z 2 ,z 3 ,...,z n }with the property that z k ∈N k−1 but z k /∈N k (and z i ≠ 0 ); this implies that thevectors in N are linearly independent. In particular, {z k ,z k+1 ,...,z n } form a basisfor N k−1 . The null vectors can be computed as follows.1. Begin with any basis for N 1 = a ⊥ 1 ,say Z 1 = {z (1)2 ,z(1) 3 ,...,z(1) n } . Becausedim N 2 = n−2 , we must have a T 2 z (1)i ≠ 0 for some z (1)i , so we can arrangethe elements of Z 1 so that a T 2 z (1)2 ≠ 0 . However, to promote numericalstability, it is preferable to invoke a “pivoting strategy” which chooses z (1)2to be the vector such that|a T 2 z (1)2 | = max2≤i≤n |aT 2 z (1)i |.Geometrically, this means that we choose from Z 1 the vector of maximumdistance from a ⊥ 2 . This is also equivalent to a form of column pivoting onA , as we shall see. If we agree to “pivot,” then it is clear that z (1)2 ∈N 1but z (1)2 /∈ N 2 .2. Successively project z (1)2 onto a ⊥ 2 through each of the remaining null vectorsz (1)3 ,z(1) 4 ,...,z(1) n to produce the new set Z 2 = {z (2)3 ,z(2) 4 ,...,z(2) n } . It canbe argued that because Z 1 is a basis for N 1 , we must have that Z 2 is abasis for N 2 . As before, dim N 3 = n−3 implies that one of these z -vectorsmust be such that a T 3 z (2)i ≠ 0 . If the pivoting strategy is again employed toselect z (2)3 such that|a T 3 z (2)3 | = max3≤i≤n |aT 3 z (2)i |,then z (2)3 ∈N 2 but z (2)3 /∈ N 3 . Again, z (2)3 is the vector in Z 2 whichmaximizes the distance from a ⊥ 3 .3. Successively project z (2)3 onto a ⊥ 3 through z (2)jfor j =4,...,n to produceZ 3 = {z (3)4 ,z(3) 5 ,...,z(3) n } . This is a basis for N 3 , and if z (3)4 is selectedto be the vector such that |a T 4 z (3)4 | = max4≤i≤n |aT 4 z (3)i | , then z (3)4 ∈N 3 butz (3)4 /∈ N 4 ..This process is repeated n − 1 times, and the last step is to project z (n−2)n−1 ontoa ⊥ n−1 through z n(n−2) to produce Z n−1 = {z n(n−1) } , which is a basis for N n−1 . Thepivot vectors N = {z (1)2 ,z(2) 3 ,...,z(n−1) n } therefore constitute a set of null vectorssatisfying z (k−1)k∈N k−1 but z (k−1)k/∈ N k .In practice, a simple normalization can be used as a safeguard against the risk ofoverflow, so the formal algorithm for computing null vectors is as follows.

a direct projection method 1161The Null Vector AlgorithmInput: Any basis {z 2 ,z 3 ,...,z n } for N 1 = a ⊥ 1 .Output: A set N of n − 1 null vectors together with a corresponding set ofn − 1 pivots.for i =2:nfor j = i : nσ j ←− a T i z jendm ←− index of max i≤j≤n |σ j |p i ←− σ m (the ith pivot)if m>i then interchange z m and z i(the ith pivot column = the ith null vector)if ‖ z i ‖ ∞ > tolerance then z i ←− z i / ‖ z i ‖ ∞if i = n go to endfor j = i +1:nz j ←− z j − σ jz ip iif ‖ z j ‖ ∞ > tolerance then z j ←− z j / ‖ z j ‖ ∞endendIf e i denotes the ith column of the identity matrix I , and if a 1k is the entry ofmaximal magnitude in a T 1 , then one choice for the initial basis for N 1 isz (1){ei if a 1i =0,i+1 = e i − a 1ie k for i ≠ k and a 1i ≠0.a 1kNotice that a 1k plays the role of the first pivot, so it makes sense to let p 1 = a 1k .Likewise, e k is the first pivot vector, and hence we let z 1 = e k . Geometrically,{z (1)2 ,...,z(1) n } is obtained projecting the unit vector e k of greatest distance froma ⊥ 1 through the other unit vectors e i , with i ≠ k .The matrixZ =[z 1 ,z 2 ,...,z n ]together with the vector of pivots p =(p 1 ,p 2 ,...,p n ) is intimately related with theLU factorization of A ; we discuss this in the next section. For now it suffices to observethat the final Z was obtained starting from the identity matrix [e 1 ,e 2 ,...,e n ] andsuccessively projecting its columns onto the null spaces N k associated with A . Thechoice of the identity matrix as the initial matrix is, in a sense, arbitrary, but is clearlyjustified in terms of simplicity and sparsity. Unless otherwise stated, it is assumedthat the null vector algorithm is started with the identity matrix. With this choice itis not difficult to see that the above algorithm returns vectors N = {z 2 ,z 3 ,...,z n }in which z i contains no more than i nonzero entries. Of course, if a T 1 is sparse, thenmost of the initial vectors will be unit columns, and if A is a sparse matrix, theneach pivot column will generally contain far fewer than i nonzero entries. This is onefacet of the technique which we will try to exploit. Furthermore, notice that this nullvector algorithm is a row-generation method in the sense that only one row of A isrequired at each step. Consequently, single rows of A can be brought from secondarymemory or else generated one at a time only as they are needed, as in Kaczmarz’siterative orthogonal projection scheme (see Censor [5]).

- Page 1: SIAM J. Sci. Comput.c○1995 Societ
- Page 5 and 6: a direct projection method 1163Clea
- Page 7 and 8: a direct projection method 1165of t
- Page 9 and 10: a direct projection method 1167For
- Page 11 and 12: a direct projection method 1169whic
- Page 13 and 14: a direct projection method 1171is p
- Page 15 and 16: a direct projection method 1173Tabl
- Page 17 and 18: a direct projection method 11756. C