Contents - Student subdomain for University of Bath

3.2. LINEAR EQUATIONS IN SEVERAL VARIABLES 65
3.2.2 Representations of Matrices
The first question that comes to mind here is “dense or sparse?”, as in definition
22 (page 31). For a dense representation of matrices, the solution is obvious:
we store a two-dimensional array (or one-dimensional array of one-dimensional
arrays if our language does not support two-dimensional arrays) containing the
values m i,j of the elements of the matrix. The algorithms for adding and multiplying
dense matrices are pretty obvious, though in fact it is possible to multiply
two 2 × 2 matrices with seven multiplications of entries rather than the obvious
eight [Str69, Win71]: this leads to being able to multiply two n × n matrices
with O(n log 2 7≈2.807 ) element multiplications rather than O(n 3 ): see Excursus
B.4.
For a sparse representation of matrices, we have more choices.
row-sparse Here M is stored as a one–dimensional array of rows: the i-th
row consisting of a list of pairs (j, m i,j ). This representation is equivalent
to that of the sparse polynomial ∑ j m i,jx j , and this technique has been
used in practice [CD85] and has a useful analogue in the case of non-linear
equations (see section 3.3.5).
column-sparse Here M is stored as a one–dimensional array of columns: the
j-th column consisting of a list of pairs (i, m i,j ).
totally sparse Here M is stored as a list of triples (i, j, m i,j ).
structured There are a large variety of special structures of matrices familar to
numerical analysts, such as ‘banded’, ‘Toeplitz’ etc. Each of these can be
stored efficiently according to their special form. For example, circulant
(a special form of Toeplitz) matrices, of the form
⎛
a 1 a 2 a 3 . . . a n−1 a n
⎞
a 2 a 3 . . . a n−1 a n a 1 a n a 1 a 2 a 3 . . . a n−1
⎜
⎝ .
. .. .
⎟
· · · · · · .. ⎠ ,
.
are, strictly speaking, dense, but only have n distinct entries, so are
“information-sparse”.
Clearly, if the matrix is structured, we should use the corresponding representation.
For randomly sparse matrices, the choice depends on what we are doing
with the matrix: if it is row operations, then row-sparse is best, and so on. One
big issue with sparse matrices is known as fill-in — the tendency for operations
on sparse matrices to yield less sparse matrices. For example, if we multiply two
n × n matrices, each with e non-zero elements per row, with n ≫ e, we would
expect, assuming the non-zero elements are scattered at random, the resulting
matrix to have e 2 non-zero elements per row. This has some apparently paradoxical
consequences. Suppose M and N are two such matrices, and we wish to
compute MNv for many vectors v. Clearly, we compute MN once and for all,