Householder's method for symmetric matrices

Numerische Mathematik 4, 354-- 361 (1962)
Handbook Series Linear Algebra*
Householder's method for symmetric matrices
Contributed by
J. H. WILKINSON
1. Theoretical background
1.1. The reduction to tridiagonal /orm
1.1.1. The symmetric matrix A 1 is reduced to the symmetric tridiagonal
matrix A~_ 1 by (n -- 2) orthogonal similarity transformations. We define matrices
A i by the relations
Ai+l=PiAiPi (i = t, 2 ..... n -- 2) (1)
where the P~ are orthogonal matrices defined by
-Pi = I -- 2 w i w T (2)
w~= Ewi,1; wi,~; ... wi,~-i; 0; 0;... ;oj (3)
~:'~=I. (4)
We assume that A i is of tridiagonal form in its last (i- t) rows and columns
and choose the elements of wi so that Ai+ 1 is of tridiagonal form in its last i
rows and columns. It is evident from the assumed form of A i and of w i that
the transformation with matrix Pi leaves the last (i- t) rows and columns un-
altered, and hence the w i have only to be chosen so that the (1, 1), (l, 2) .....
(l, 1 -- 2) elements of Ai+ 1 are zero (l=n -- i + 1). Clearly all the A i are symmetric.
There are two essentially different determinations of the vector w i I1~, [21
and [31, and these we now describe. In defining the i-th transformation we
denote the (j, k) element of A i by aik, omitting for convenience reference to
the suffix i. Defining ai and H i by the relations
the two determinations of wi are given by
ai=a~.l+a , 2 l,z+.,.+au_1 (S)
Hi =ai :E al, l-1 a} (6)
Wi, l_l= Eal, l_t ~ai]/(2Hi) 89 (7)
wi, j=au/(2H3~ i =t, 2 ..... l --2. (8)
* Editors note. In this fascicle, prepublication of algorithms from the Linear
Algebra series of the Handbook for Automatic Computation is started. Algorithms
are published in ALGOL 60 reference language as approved recently by IFIP.
[BACKUS, J. W., BAUER F. L. et al. : Report on the algorithmic language ALGOL 60.
Numer. Math. 2, 106-- t 36 (1960)]. Contributions in this series should be styled after
the most recently published ones. Inquiries are to be directed to the editor.
F. L. BAUER, Mainz.

If we write
then
J. H. ~u : Householder's method ~or symmetric matrices 3 5 5
u~= [al,1; al,~; ...; al,,-2; ai, l-1 0;...; O] (9)
I- 2w, w~ = I ~' ~ (to)
Hi
1.1.2. In the above formulation it is assumed that we take either the upper
signs or the lower signs throughout. In practice the choice must be made so
as to give numerical stability and this is achieved by taking the plus sign in
equation (6) if at, l_l>=O and the minus sign otherwise.
It is important to take advantage of the symmetry of both Ai+ ~ and A~
when evaluating Ai+ 1 from equation (1), and to this end we compute only the
elements of Ai+ 1 on and below the diagonal. A convenient way of organising
the computation is as follows. We write
~ A~ u~+ (11)
----- A ~ -- u i H~ H~ H~
This may be simplified by introducing vectors p~ and qi and a scalar K i defined by
then we have
p~ =A~ udH~ (t2)
Ki=u~pd(2H3 (t3)
qi=P~-- Ki u~ (t4)
Equation (t5) is used only to compute the elements of rows (/) to (l--t) of
A~+~ on and below the diagonal. The elements of rows (l+t) to (n) are the
same as in A i; in row l the (1, l) element is the same as in A i while the (l, l -- t)
element becomes T a~.
From the form of A~ and u i it is evident that Pi is of the form
so that q~ is given by
PT=- EP~,~; P~,~; ..." P~,~-~; P~,~; 0; ...; o~
q~-~ [Pi, x --Kiui,1; Pi,2 --Kiui,2; ... ; Pi, l-x- Kiui, l-1; P~,z; 0; ... ; 0~.
If a~ is zero then we may take P~=I and no transformation is needed. This
case is given special treatment. We could in fact take P~= I if
a~,x + at,~ + ... +a~,~_~=0
since the l-th row and column are then already in tridiagonal form, but this
has not been done. Note that in this case the transformation with matrix P~
merely changes the sign of the (l, l -- ~) element.

356 Handbook Series Linear Algebra
1.2. Backtrans/ormation
If z~_ 1 is an eigenvector of the tridiagonal matrix An_ 1 derived by the
Householder process, then the corresponding eigenvector of A 1 is P~ P2 -.. P,-2z~-1 -
so that
We write
Pk Pk+l ... Pn-2Zn-l=Zk (1)
Pkzk+l = zk. (2)
From equations (6), (9) and (t0) of the Householder method we have
u k u T
Zk=Zk+~ Hk zk+l. (3)
The non-zero element of u k are stored in the /-th row of the lower triangle of
the array A, (l----n--k+ Q, and H~ may be computed from the relation
-- H k -=:q: a~ [at, z_ 1 -r (4)
The first factor on the right of equation (4) is stored in bk and the second as
the (1, l--t) element of A. The right-hand side of (3) is computed in the steps
U T gk+ 1
S -- Hk , (5)
zk =zk+l + Suk. (6)
Each of the z k is overwritten in the store occupied by the original z~_ 1 from
which it was produced so that no extra storage is required in the back-trans-
formation.
2. Applicability
The algorithm householder tridiagonalization works for arbitrary symmetric
matrices, possibly with multiple eigenvalues. In the latter case, the resulting
tridiagonal matrix would decompose if the computation were exact. In practice
we will expect that it will "almost decompose".
The algorithm backtrans/ormation derives the eigenvector system of A from
an eigenvector system of the tridiagonal matrix.
3. Formal parameter list
The symmetric matrix A of order n is given as a lower triangular matrix.
It is reduced to a symmetric tridiagonal matrix, the diagonal elements of which
are stored as a vector c and the subdiagonal elements as a vector b (with an
additional last component b~= 0). The lower triangle of A together with b are
used to store sufficient information for the details of the transformation to be
recoverable in the procedure backtransformation.
Backtransformation: The input matrix z contains the ml eigenvectors of
the tridiagonal matrix which were computed by the procedure tridiinverse
iteration, (zii is the /-th component of the eigenvector corresponding to wj).
At the end of the procedure z is the matrix of the corresponding eigenvectors
of the initial matrix A.

j. ]7I. \VILKINSON : Householder's method for symmetric matrices 357
4. Algol programs
procedure householder tridiagonalisation (a, n) result: (c, b, a);
value n;
integer n;
array a, b,
comment
c;
The symmetric matrix a of order n is given as a lower triangular
matrix. It is reduced to a symmetric tridiagonal matrix, the diagonal
elements of which are stored as the vector c and the sub-diagonal
elements as the vector b (with an additional last component b In I = 0).
The lower triangle of a together with b are used to store sufficient
information for the details of the transformation to be recoverable
in the procedure backtrans/ormation;
begin
integer i, i, k;
real ai, sigma, h, b?', bigk, bi ;
array q El : n -- t ~ ;
for i ,=n step -- t until 3 do
begin sigma :=0;
for k,=l step t until i--t do
sigma :=sigma + a Ei, k I a[i, k~ ;
ai:=a[i,i-- ~;
if ai >-- 0 then bi =-- sqrt (sigma)
else bi := sqrt (sigma);
if bi ~ 0 then
begin
h :=sigma -- ai bi;
a[i, i--1~ =ai--bi;
forj =i--1 step --1 until 1 do
begin bi :=0;
for k ,=i -- t step -- 1
until j do
bi ,= bi + a [k, i~ a [i,k ~ ;
for k =i -- l step -- l
until t do
bi ,=hi+ all", k~ k];
q [i~ '=bi/h
end?';
bigk ,= 0;
Numer. Math. Bd. 4
comment
comment
Selection of stable sign;
Special case sigma = 0:
transformation skipped;
comment Here normalised addition
must be used otherwise
orthogonality of the trans-
formation most likely will
be lost in the case of can-
cellation;
comment Computation of A
comment Computation of p;
25

358 Handbook Series Linear Algebra
fori :=i-- 1 step -- 1 until 1 do
bigk ,=bigk + a [i, ]] x q [1'] ;
bigk ,=bigk/(2 x h) ;
comment Computation of K;
for i ,=i -- 1 step -- 1 until I do
q [/'] ,=q [/'] -- bigk X a [i,/'~ ;
for/' ,=i -- 1 step -- t until t do
begin
for k :=j step -- t until t do
comment Computation of q;
,~[i, k] ,=a[i, k~-,~[i,i] x q [k~-a [i, k] x~,[i];
comment Computation of the (i + 1)th
matrix A ;
endj
end
end i;
fori :--n step -- t until t do
c [i] :=a[i, i];
bell ,----a[2, 1] ;
b In] ".--0
end ;
procedure backtrans/ormation (a, b, z, n, ml) result: (z);
value n, ml ;
integer n, ml;
array a, b, z;
comment b is the subdiagonal of the symmetric tridiagonal matrix of order n.
z is the matrix formed by ml eigenvectors of the tridiagonal matrix.
The corresponding matrix of eigenvectors of the original matrix is
computed by a backtransformation and overwritten on z. The details
of the transformation are stored in the lower triangle of a and in b;
begin
integer i, i, k;
real s;
for/' ,=1 step t until m] do
fork =3 step I until n do
ifb [k -- 1] ~0 then
begin s :=0;
fori :=1 step I until k -- 1 do
s ,=s+a[k,i] iJ;
s =s/(b [k - t~ x ~[k, k - 1]);
fori ,=t step t until k--t do
zEi, il ,=~ [i,i] +s x,~ [k, i]
end
end;
5. Organizational and notational details
5.1. Householder tridiagonalisation
In the ALGOL procedure it is assumed that A z is given originally as the
lower triangle of an n array. The reduction to tridiagonal form is made

J. H. WILKINSON: Householder's method for symmetric matrices 359
working with this lower triangle of elements, the details of the transformation
being overwritten on the same triangle. From equations (6) and (10) it is clear
that we will retain full information on each transformation if we store the com-
ponents of the u i and :J: a, ~. Each non-zero element of each u i is overwritten
on the ali element from which it is derived. This is particularly convenient
because apart from the elements ui, l_ 1 each uii is equal to the corresponding
current a~i.
The storage of the elements of each u i leaves no space available for the
(1, l--t) element of each Ai+ 1. These elements are, of course, the off-diagonal
elements of the final tridiagonal matrix. They are accordingly stored in the
linear array b of order n. An array q is also used to store the components of
the successive Pi and qi- The vector qi is overwritten on Pi-
The off-diagonal elements of A~_I are obtained finally as the linear array b.
A copy of the diagonal elements of A~_ 1 is made to give the linear array c. This
is convenient since tridiagonal matrices often arise as primary data and it is
obviously desirable to present such matrices as linear arrays rather than as a
square array.
5.2. Backtrans/ormation
The matrix z containing the eigenvectors of the tridiagonal matrix is over-
written by the corresponding matrix of the eigenvectors of the original matrix.
The matrix A is equal to the matrix resulting from a Householder tridiagonali-
sation, but the process uses only its lower triangle.
6. Discussion of numerical properties
In determining the (1 -- t) element of the vector w i it is important to get exact
orthogonality of the transformation even when cancellation occurs, therefore
normalized addition must be used there.
7. Examples of the use of the procedure
householder tridiagonalisation may be the first step in the determination
of the eigenvalues and eigenvectors of the matrix A followed by a determination
of the eigenvalues by the procedure tridibisection and possibly by a determination
of the eigenvectors of the resulting tridiagonal matrix by the procedure tridi-
inverse iteration. In the last case the procedure backtrans/ormation is to be
applied.
householder tridiagonalisation may also be the preparation for the appli-
cation of the QD or QR algorithm.
8. Test results
The procedures were tested on the Z 22 and Siemens 2002 in the Institute
for Applied Mathematics at Mainz University and a second test was performed
in the Institute for Applied Mathematics of the EidgenSssische Technische
Hochschule at Zfirich.
We give two examples, the second being of a matrix for which the correspond-
ing tridiagonal matrix decomposes into the direct sum of two smaller tridiagonal
25*

360 Handbook Series Linear Algebra
matrices. The results of the tridiagonalisation on Siemens 2002 are displayed.
For the second example, the tridiagonal matrix obtained on a computer with
a different word-length (or even with the same word-length but with a different
rounding procedure) may differ completely in its upper segment. The eigen-
values should be unaffected.
The eigenvectors of the two tridiagonal matrices were computed by tridibi-
section (see article "Calculation of the eigenvectors of a symmetric tridiagonal
matrix by inverse iteration", section 8). We give these vectors and the eigen-
vectors of the original matrix A; the latter were computed by the procedure
backtrans/ormation.
Lower triangle of Lower triangle of
matrix A matrix B
5; 1;
4; 6; 2; 4;
3; O; 7; 3; 5; 6;
2; 4; 6; 8; O; --t; --2; 1;
t; 3; 5; 7; 9; 1; o; --3; 2; 4;
2; 3; O; 3; 5; 6;
Matrix A
b[i] c[i]
--3.290140684 1.380806052
1.901233318 4.029608282
3.485636221 5.161014238
--9.165151390 1.5428571441o I
0 9.000000000
Eigenvectors of the matrix A
--2.4587793861o--t
--3.0239603961o--I
--4.532t452331o--1
--0.577177153
-- 5.56384583 71o-- 1
-- 5.5096195601o-- 1
--0.709440339
3.40t 79t3251o--t
8.34t095344~o--2
2.6543567721o-- 1
Matrix B
b[q c[q
--0.965478361 --0.442587832
1.894086692 --0.693465710
--1.1663618651o--8 1.2136053561oi
--1.000000002 0.000000000
--6.855654601 5.000000000
0 6.000000000
Eigenvectors of the matrix B
--2.21789750510-- I
--4.729t t32991o-- l
--0.720938140
2.5941489041 o- 1
3.5780708731o--t
1.0995626711o-- 1
t .70061797910-- 1
t.7858463011o-- 1
--t .3806649231o--t
2.9591512961o--1
0.565489671
7.1608646531o-- t

J. H. WILKINSON: Householder's method for symmetric matrices 36t
Eigenvectors of the matrix A
-- 5.47t 727952ao-- f
3.12569920610-- t
--0.618112077
1 A 560659371o--t
4.5549374641o-- 1
3.4to1304181o--I
- - t. 1643462OOlo-- t
--o.1959o67121o--1
--o.682o43o35
6.36o71214Olo -- t
4.6935807241o-- t
--0.542212195
- - 5.4445240331o-- t
4.2586566t91o--1
8.8988503651o--2
R e f e r e n c e s
Eigenvectors of the matrix B
6.6954556761o--t
--3.9533t735010--t
1.36726362t io-- 1
--0.288372768
4.63372192810 -- t
- - 2.808 t 098561o-- 1
0.t3164t89tlo--t
2.5928612341o-- 1
--1.995t 543031o--t
--0.728887389
5.5tt 5485651o--t
--2.40296694tlo--t
5.0395 t 79731o- 1
o. 740322901 lo-- t
- - 5.2916056301o-- t
--3.t320230781o--t
-- 5.2138969241o-- t
3.0o99502941o--t
3.9101568851o--1
--0.8087821031o--t
--4.t 868566571o-- t
- - 4.4628447191o--
--0.520371701
4.4t 94029251o-- t
I1] BAUER, F. L., and HOUSEHOLDER, A. S. : On certain methods for expanding the
characteristic polynomial. Numer, Math. 1, 29-- 37 (t 959).
[2] BAUER, F. L.: Sequential reduction to tri-diagonal form. J. Soc. Indust. Appl.
Math, 7, 107--113 (t959).
[3] WILKINSON, J. H. : Householder's method for the solution of the algebraic eigen-
problem. Computer J. 3, 23 (1960).
National Physical Laboratory
Teddington, Middlesex