Iterative Methods for Sparse Linear Systems Second Edition

10 CHAPTER 1. BACKGROUND IN LINEAR ALGEBRA
1.6 Subspaces, Range, and Kernel
A subspace of C n is a subset of C n that is also a complex vector space. The set of
all linear combinations of a set of vectors G = {a1, a2, . . .,aq} of C n is a vector
subspace called the linear span of G,
span{G} = span {a1, a2, . . .,aq}
= z ∈ C n
q
z =
i=1
αiai; {αi}i=1,...,q ∈ C q
If the ai’s are linearly independent, then each vector of span{G} admits a unique
expression as a linear combination of the ai’s. The set G is then called a basis of the
subspace span{G}.
Given two vector subspaces S1 and S2, their sum S is a subspace defined as the
set of all vectors that are equal to the sum of a vector of S1 and a vector of S2. The
intersection of two subspaces is also a subspace. If the intersection of S1 and S2 is
reduced to {0}, then the sum of S1 and S2 is called their direct sum and is denoted
by S = S1 S2. When S is equal to Cn , then every vector x of Cn can be written
in a unique way as the sum of an element x1 of S1 and an element x2 of S2. The
transformation P that maps x into x1 is a linear transformation that is idempotent,
i.e., such that P 2 = P . It is called a projector onto S1 along S2.
Two important subspaces that are associated with a matrix A of Cn×m are its
range, defined by
Ran(A) = {Ax | x ∈ C m }, (1.17)
and its kernel or null space
Null(A) = {x ∈ C m | Ax = 0 }.
The range of A is clearly equal to the linear span of its columns. The rank of a
matrix is equal to the dimension of the range of A, i.e., to the number of linearly
independent columns. This column rank is equal to the row rank, the number of
linearly independent rows of A. A matrix in C n×m is of full rank when its rank is
equal to the smallest of m and n. A fundamental result of linear algebra is stated by
the following relation
C n = Ran(A) ⊕ Null(A T ) . (1.18)
The same result applied to the transpose of A yields: C m = Ran(A T ) ⊕ Null(A).
A subspace S is said to be invariant under a (square) matrix A whenever AS ⊂
S. In particular for any eigenvalue λ of A the subspace Null(A − λI) is invariant
under A. The subspace Null(A − λI) is called the eigenspace associated with λ and
consists of all the eigenvectors of A associated with λ, in addition to the zero-vector.
.