CHAPTER II DIMENSION In the present chapter we investigate ...

CHAPTER II
DIMENSION
In the present chapter we investigate interrelations between the notions of linear independence,
spanning sets, bases and dimensions, which are central to the basic theory of
linear algebra. Among other things, we establish the crucial identity relating the dimensions
of V , T (V ) and ker T for a linear transformation T defined on V .
§1. Linear Independence
1.1. Linear independence (or its opposite, linear dependence) is by far the most
important concept in describing relations among vectors. A thorough understanding is a
“must” for working with linear algebra. The reader is advised to study this concept very
carefully, and to contemplate its meaning as many times as possible.
Consider a finite set of vectors v1, v2, . . . , vr in a linear space V . By a linear
relation among these vectors we mean an identity of the form
a1v1 + a2v2 + · · · + arvr = 0. (1.1.1)
where a1, a2, . . . , ar are some scalars. We give a quick example: the linear relation
2v1 + v2 + (−1)v3 + 0 v4 = 0 holds for v1 = (1, 1), v2 = (1, 0) v3 = (3, 2) and
v4 = (5, 7), as we can check. (2.1.1) is certainly true when all ak (1 ≤ k ≤ r) are zeros:
0 v1 + 0 v2 + · · · + 0 vr = 0 (1.1.2)
holds trivially. Let us call (1.1.2) the trivial linear relation among v1, v2, . . . , vr. If the
linear relation given as (2.1.1) is nontrivial, that is, it is not of the form (1.1.2), then at
least one of the scalars a1, a2, . . . , ar is nonzero, that is, ak = 0 fo some k.
Definition. W e say that vectors v1, v2, . . . , vr are linearly dependent if there is a
nontrivial linear relation among them, that is a1v1 + a2v2 + · · · vr = 0 holds for some
a1, a2, . . . , ar, not all equal to zero.
To make our language flexible, we also say a set of vectors v1, v2, . . . , vr is linearly
dependent if vectors v1, v2, . . . , vr are linearly dependent.
1

In order to prove that a given set of vectors v1, v2, . . . , vr is linearly dependent,
we may write down a1v1 + a2v2 + · · · · · · + arvr = 0 and treat it as an equation with
a1, a2, . . . , ar as unknowns and try to find a nontrivial solution of it. To give a quick
example, let us check if the vectors v1 = (i, 1), v2 = (1, −i) in C 2 are linearly dependent.
We begin by writing down a1v1 + a2v2 = 0, or a1(i, 1) + a2(1, −i) = (0, 0). which gives
ia1 + a2 = 0 and a1 − ia2 = 0. This system of equations in a1, a2 has a nontrivial solution,
such as a1 = i and a2 = 1. So the given vectors are linearly dependent.
The opposite of linear dependence is linear independence. A set of vectors is linearly
independent if this set is not linearly dependent. Recall that v1, v2, . . . , vr are linearly
dependent if there is a nontrivial linear relation among them. So “vectors v1, v2, . . . , vr
are linearly independent” means that there is no nontrivial linear relation among them, in
other words, the only possible linear relation among them is the trivial one. Thus, if we
have a linear relation a1v1 + a2v2 + · · · + arvr = 0 among linearly independent vectors
v1, v2, . . . , vr, then all ak (0 ≤ k ≤ r) must be zeroes. Thus we have arrived at
Definition. We say that vectors v1, v2, . . . , vr are linearly independent if any
linear relation among them, say
a1v1 + a2v2 + · · · · · · + arvr = 0,
is necessarily trival, that is, a1 = a2 = · · · = ar = 0.
To prove that given vectors v1, v2, . . . , vr are linearly independent, we almost always
start with something like
“Suppose we have a1v1 + a2v2 + · · · + arvr = 0.”
After that, we try to figure out why a1, a2, . . . , ar all must be equal to zero. In the next
subsection, we give a few examples to see how it works.
1.2. First we give a quick example to show the steps. We are asked to prove that
vectors v1 = (1, 0), v2 = (1, 2) in R 2 are linearly independent. We begin with the
sentence “Suppose that a1v1 +a2v2 = 0”. Then we proceed by rewriting a1v1 +a2v2 = 0
as a1(1, 0) + a2(1, 2) = (0, 0), or (a1 + a2, 2a2) = (0, 0), which gives us a1 + a2 = 0 and
2a2 = 0, from which we deduce a1 = a2 = 0. Here we emphasize that the first sentence in
our proof must be correct, no matter how hard or how easy the given problem is.
Example 1.2.1. Prove that vectors u and v in a vector space V are linearly independent
if u + v and u − v are linearly independent.
2

♠ Aside. What is supposed to be proved? Well, the linear independence of u and v.
So we should start with something like “Assume au + bv = 0”. Then we try to derive
a = 0 and b = 0. A common error is to start with “Assume a(u + v) + b(u − v) = 0”
which results in receiving no credit for the rest of the work. ♠
Solution. Assume au + bv = 0. Let p = u + v and q = u − v. Then, by assumption,
p and q are linearly independent. Notice that 2u = p + q and 2v = p − q. Hence
0 = 2(au + bv) = a(2u) + b(2v) = a(p+q) + b(p−q) = (a+b)p + (a−b)q.
Since p, q are linearly independent, a + b = 0 and a − b = 0. So a = b = 0. Done.
Example 1.2.2. Let T be a linear operator on V and let v be in V . Assume that
T 3 v = 0 but T 2 v = 0. Prove that v, T v, T 2 v are linearly independent.
Solution. Assume
a1v + a2T v + a3T 2 v = 0. (1.2.1)
Applying T to this identity, we have T (a1v + a2T v + a3T 2 v) = T 0 = 0. Rewrite the last
identity as a1T v + a2T 2 v + a3T 3 v = 0. Since T 3 v = 0, we have
a1T v + a2T 2 v = 0. (1.2.2)
Applying T to the last identity again, we have a1T 2 v + a2T 3 v = 0, or a1T 2 v = 0. Since
T 2 v = 0, we must have a1 = 0. Thus (1.2.2) becomes a2T 2 v = 0. Since T 2 v = 0, we
must have a2 = 0. Now (1.2.1) becomes a3T 2 v = 0. Since T 2 v = 0, we must have a3 = 0.
We have shown that a1 = a2 = a3 = 0. Hence v, T v, T 2 v are linearly dependent.
Remark. More generally, if T m v = 0 and T m+ 1 v = 0, then the vectors
vm = v, vm−1 = T v, vm−2 = T 2 v, . . . , v1 = T m−1 v, v0 = T m v
are linearly independent. This remark will be refered to in the future and the labeling of
vectors here is competible to the discussion there.
Example 1.2.3. Let T be a linear operator on a vector space V and let u and v
be nonzero vectors in V such that T u = 2u and T v = 3v. Prove that u, v are
linearly independent.
Solution. Assume
au + bv = 0. (1.2.3)
3

Applying T to this identity, we have T (au + bv) = 0, or aT u + bT v = 0. From
T u = 2u and T v = 3v we get
2a u + 3b v = 0. (1.2.4)
Multiply (1.2.3) by 2 and then substract (1.2.4), we get bv = 0. Since v is nonzero, we
must have b = 0. Thus (1.2.3) becomes au = 0. Since u is nonzero, we must have a = 0.
We have shown a = b = 0. Hence u, v are linearly independent.
Example 1.2.4. Let c1, c2, . . . , cn, cn+ 1 be n + 1 distinct complex numbers and con-
) for k = 0, 1, 2, . . . , n:
sider vectors in C n+ 1 : vk = (c k 1 , ck 2 , . . . , ck n , ck n+ 1
v0 = (1, 1, 1, . . . , 1, 1)
v1 = (c1, c2, . . . , cn, cn+ 1)
v2 = (c 2 1 , c2 2 , . . . , c2 n , c2 n+ 1 )
.
vn = (c n 1 , cn 2 , . . . , cn n , cn n+ 1 )
Prove that v0, v1, v2, . . . , vn are linearly independent.
Proof: Assume a0v0 + a1v1 + · · · + anvn = 0. For each j with 1 ≤ j ≤ n + 1, write
out the jth component of this vector identity:
a0 + a1cj + a2c 2 j + · · · + anc n j
This shows that the polynomial a0 +a1x+a2x 2 +· · ·+anx n has n+1 roots c1, c2, . . . , cn+ 1.
But a polynomial of degree at most n cannot have n + 1 roots, unless that polynomial is
the zero polynomial. Therefore a0, a1, . . . , an must vanish. This shows v0, v1, v2, . . . , vn
are linearly independent.
1.3. Given a finite set of vectors v1, v2, . . . , vr in V , we can construct a subspace
containing these vectors by collecting all linear combinations of these vectors. Here, by a
linear combination of v1, v2, . . . , vr we mean a vector of the form
α1v1 + α2v2 + · · · · · · + αrvr
= 0.
(1.3.1)
for certain scalars α1, α2, . . . , αr. A quick example: 4 + 2x − x 2 is a linear combination of
2 + 3x and 4x + x 2 because 4 + 2x − x 2 = 2(2 + 3x) + (−1)(4x + x 2 ), which can be easily
checked. (For the moment let us not worry about how to figure out this identity.) One
can think of a linear combination of the form (1.3.1) as a “combo soup” with vk as its
kth ingredient and ak as the amount of this ingredient (1 ≤ k ≤ r).
4

The set L of all linear combinations of given vectors v1, v2, . . . , vr is indeed a subspace
of V . To prove this, let us take two such linear combinations, say
v = α1v1 + · · · + αrvr and w = β1v1 + · · · + βrvr.
We have to show that αv + βw is also in L, where α and β are arbitrary scalars. Now
αv + βw = (αα1 + ββ1)v1 + · · · + (ααr + ββr)vr,
which is again a linear combination of the given vectors. Hence αv + βw ∈ L.
This furnishes the proof of L being a subspace. This subspace is called the (linear)
span of the set {v1, v2, . . . , vr}, or the subspace spanned by v1, v2, . . . , vr. You may write
L = span{v1, v2, . . . , vr}.
Thus, we say that vectors v1, v2, . . . , vr span the whole space V , or they form a spanning
set if V = span{v1, v2, . . . , vr}, that is, every vector in V is a linear combination of
v1, v2, . . . , vr. A vector space V is said to be finite dimensional and we write dim V < ∞
if it is spanned by a finite set of vectors, otherwise it is said to be infinite dimensional
and we write dim V = ∞.
A subspace of V can be described as a nonempty set W of V closed under taking
linear combinations, that is, linear combinations of vectors in W are also vectors in W .
The span of vectors v1, v2, . . . , vr in a vector space V can be described as the smallest
subspace containing v1, v2, · · · , vr.
Recall from subsection 3.5 of Chapter I that, given a linear operator T on a vector
space V , a subspace M of V is said to be invariant for T if T (M) ⊆ M, that
is, every vector v in M is sent by T to a vector (namely T v) in M. In case M is
spanned by v1, v2, . . . , vr, that is, M = span{v1, v2, . . . , vr}, to show that T (M) ⊆ M, it
is enough to check that T vk is in M for k = 1, 2, . . . , r. Indeed, if v is in M, then we can
write v = a1v1 + a2v2 + · · · + arvr for some scalars a1, a2, . . . , ar and hence
T v = a1T v1 + a2T v2 + · · · + arT vr
is in M because it is a linear combination of vectors T v1, T v2, . . . , T vr which are in M.
Example 1.3.1. Let F(R) be the space of differentiable functions of one variable x.
Define operators D and Ta (a here is any real number) by putting D(f(x)) = f ′ (x) and
Ta(f(x)) = f(x + a). Here D is a differential operator and Ta is a translation operator.
5

They are among the most important operators used in differential equations and difference
equations. We can check that each of the following (finite dimensional) subspaces is
invariant for both D and Ta:
V1 = span {1, x, x 2 }, V2 = span {e kx , xe kx , x 2 e ax }, V3 = span {e kx cos bx, e kx sin bx},
V4 = span {cos bx, sin bx, x cos bx, x sin bx, x 2 cos bx, x 2 sin bx}.
Here k and b are any real constants. Here we only check that V3 is invariant for both
D and Ta and we leave the rest of verification to the reader as an exericse; (see Drill 7).
Indeed, by the product rule in differentiation, we have
D(e kx cos bx) = D(e kx ) cos bx + e kx D(cos bx) = ke kx cos bx − be kx sin bx
D(e kx sin bx) = D(e kx ) sin bx + e kx D(sin bx) = ke kx sin bx + be kx cos bx
which are linear combinations of e kx cos bx and e kx sin bx and hence are in V3. This proves
D(V3) ⊆ V3. Next,
T (e kx cos bx) = e k(x+ a) cos(b(x + a)) = e kx+ ka cos(ab + bx)
= e ak e kx (cos ab cos bx − sin ab sin kx)
= e ak cos ab(e kx cos bx) − e ak sin ab(e kx sin kx)
which is a linear combination of e kx cos bx and e kx sin bx and hence are in V3. Similarly
we can check that T (e kx sin bx) is in V3. Hence Ta(V3) ⊆ V3. (We recommend the reader
to review Examples 3.5.1 to 3.5.4 in the last chapter. In each of these examples the choice
of the invariant subspace is suggested by our present example.)
If v is a linear combination of a set of linear independnt vectors v1, v2, . . . , vr,
then this linear combination must be unique. Indeed, suppose we have both
v = α1v1 + α2v2 + · · · + αnvr and v = β1v1 + β2v2 + · · · + βnvr.
Subtract these identities:
0 = v − v = (α1 − β1)v1 + (α2 − β2)v2 + · · · + (αr − βr)vr.
Since, by assumption, v1, v2, . . . , vr are linearly independent, we must have α1 − β1 = 0,
α2 − β2 = 0, etc. and hence α1 = β1, α2 = β2, etc. Using the argument presented here,
we can prove: a set of vectors form a basis of V if and only if: 1. this set is linearly
independent, and 2. it spans V . Recall from Definition 2.6.1 that vectors v1, v2, . . . , vn
in V form a basis of V if and only if each vector v can be uniquely written as a linear
6

combination of them, that is, v can be written as a1v1 + a2v2 + · · · + anvn, where the
scalars a1, a2, . . . , an are uniquely determined by v.
1.4. Let us take an arbitrary finite set of vectors v1, v2, . . . , vr in a linear space V .
For each positive integer k with k ≤ r, let Vk be the span of v1, v2, . . . , vk, that is,
Vk = span {v1, v2, . . . , vk}.
For convenience, let V0 = {0}, the trivial subspace. So
V0 = {0}, V1 = span {v1}, V2 = span {v1, v2}, . . . Vr = span {v1, v2, . . . , vr}.
Clearly the family of subspaces Vk is increasing:
{0} = V0 ⊆ V1 ⊆ V2 ⊆ · · · ⊆ Vr−1 ⊆ Vr.
(That Vk ⊆ Vk+ 1 means that Vk is contained in Vk+ 1, that is, every vector in Vk is also
a vector in Vk+ 1.) For each positive integer k with 1 ≤ k ≤ r, there are two possibilites
Case 1: Vk−1 = Vk. This happens exactly when vk is in Vk−1.
Case 2: Vk−1 = Vk. This happens exactly when vk is not in Vk−1.
When the second case occurs, we call vk a leading vector. Thus vk is a leading vector
if and only if vk is not in the span of the preceding vectors, namely v1, v2, . . . , vk−1.
Now we claim
Theorem 1.4.1. W ith the notation and terminology as above, the we have
(a) leading vectors are linearly independent, and
(b) any non-leading vector is a unique linear combination of preceding leading vectors.
We postpone the proof of this theorem to the end of the present section. Here let us
draw some important conclusions from it. First, let us take a spanning set of vectors
v1, v2, . . . , vr in V . From part (a) of the above theorem, we see that the leading vectors
in this set are linearly independent. From part (b) we see that all vectors vk (1 ≤ k ≤ r)
in this spanning set are linear combinations of leading vectors. This shows that leading
vectors also span V . Now the leading vectors are both linearly independent and spanning.
Hence the leading vectors form a basis of V . We have proved
Corollary 1.4.2. Any finite set of vectors spanning a linear space V contains a subset
of vectors that form a basis of V .
7

As a result of this corollary, we know:
Corollary 1.4.3. Every finite dimensional vector space has a basis.
Next, assume that a set of linearly independent vectors w1, w2, . . . , wp in a finite
dimensional space V is given. Let v1, v2, . . . , vr be any spanning set of V . Now
put these two sets of vectors together to form
w1, w2, . . . , wp, v1, v2, . . . , vr
The above list of vectors certainly spans V . By our previous argument, we see that the
leading vectors in this list form a basis of V , say B. Since the linearly independent vectors
w1, w2, . . . , wp appear at the beginning of this list, all of them are leading vectors. So
they belong to the basis B. we have shown
Corollary 1.4.4. A finite set of linearly independent vectors in a finite dimensional
linear space V can be enlarged to a basis of V .
1.5. Given a finite set of vectors v1, v2, . . . , vn in a finite dimensonal linear space
V , how do we find the leading vectors and how do we express other vectors as linear
combinations of the leading vectors, as suggested by Theorem 1.4.1 above?
First, by using a coordinate system, we can convert these vectors into column vectors
in space F m (where F is the field we work with and m is the number of coordinates),
say C1, C2, . . . , Cn. In this way we turn the original problem about vectors into the one
about column vectors. We form an m × n matrix A by using these vectors as columns:
A = [C1 C2 · · · Cn]
Now we apply elementary row operations to this matrix to its reduced row echelon form.
Recall that elementary row operations either exchange two rows, or adding a scalar multiple
of one row to another, or multiply one row by a nonzero scalar. An elementary row
operation has the same effect as multiplying an invertible matrix on the left. For example,
given a 2 × 2 matrix with A, exchanging two rows (adding 2 × the second row to the first,
resp.) has the same effect as multiplying A on the left by
as we can check that
0
1 a
b
1 0 c d
=
0 1
1 0
c d
,
a b
(by
1 2
0 1
resp.)
1 2 a b
=
0 1 c d
8
a + 2c b + 2d
.
c d

Applying elementary row operations several times to a m×n matrix A, we end up with an
echelon matrix, say B. The effect of these operations is the same as multiplying A on the
left consecutively by some invertible matrices, say E1, E2, . . . , Eℓ, that is, B = EA where
E = Eℓ . . . E2E1. As a product of invertible matrices, E is invertible. Now
B = EA = E[C1 C2 · · · Cn] = [EC1 EC2 · · · ECn]
This identity tells us that EC1, EC2, . . . , ECn are columns of B. We claim:
Theorem 1.5.1. If matrices A and B are row equivalent and if a linear relation holds
among columns of A, then the same relation holds for corresponding columns of B.
More precisely, this theorem says that if the linear relation
holds, then so does the linear relation
α1C1 + α2C2 + · · · + αnCn = O
α1EC1 + α2EC2 + · · · + αnECn = O.
But this is completely obvious! Just multiply the first identity on the left by E! With the
help of this theorem, the question raised at the beginning of §1.5 is answered, as illustrated
in the next example.
Example 1.5.2. Consider the following “vectors” in P3:
p1(x) = 1 + 3x + 4x 2 + 2x 3 , p2(x) = 2 + 6x + 8x 2 + 4x 3 ,
p3(x) = 2 + x + x 2 + x 3 , p4(x) = 1 + x 2 + x 3 , p5(x) = 9 + 6x + 9x 2 + 7x 3 .
We are asked to find the leading vectors and express each vector as a linear combination
of preceding leading vectors.
Solution. Use the natural basis {1, x, x 2 , x 3 } we convert the given vectors into
columns A1, . . . , A5 and form the matrix A = [A1 A2 A3 A4 A5] Then reduce A to
its reduced row echelon form B = [B1 B2 B3 B4 B5]. The actual computation shows
⎡
1 2 2 1
⎤
9
⎡
1 2 0 0
⎤
1
⎢ 3
A = ⎣
4
6
8
1
1
0
1
6 ⎥
⎦ .
9
⎢ 0
B = ⎣
0
0
0
1
0
0
1
3 ⎥
⎦ .
2
2 4 1 1 7
0 0 0 0 0
The leading column vectors in B are B1, B3, B4, and B2 = 2B1, B5 = B1 + 3B3 + 2B4.
By Theorem 1.5.1, we know that in A, the leading column vectors are A1, A3, A4, and
9

A2 = 2A1, A5 = A1 + 3A3 + 2A4. Hence, among the given “vectors” in P3, the leading
vectors are p1(x), p3(x), p4(x) and p2(x) = 2p1(x), p5(x) = p1(x) + 3p3(x) + 2p4(x).
Remark. As we know, any matrix A can always be row reduced to a reduced row
echelon matrix, say B. However, row reduction can be performed in many different ways.
Naturally a question is raised: is B uniquely determined by A? The answer is Yes. The
reason is, first, those columns in B containing leading ones correspond to leading column
vectors of A, second, the entries of any other column in B is determined by the way it is
written as a linear combination of columns containing leading ones, the same way as the
corresponding column in A expressed as a linear combination of leading vectors, and such
a linear combination is unique because the leading vectors are linearly independent. The
uniqueness of reduced row echelon form is considered by some people as a “hard theorem”.
In fact, from our view it is easy to understand.
1.6. Now we return to the proof of Theorem 1.4.1. Let the leading vectors be
vk1 , vk2 , . . . , vkp , where 1 ≤ k1 < k2 < · · · < kp ≤ r. Suppose we have
a1vk1 + a2vk2 + · · · + apvkp = 0. (1.6.1)
If all coefficients a1, a2, . . . , ap in (1.6.1)are zeroes, then there is nothing to prove. So
we assume that one of them is nonzero. Let q be the largest positive number for which aq
is nonzero. Then (1.6.1) becomes
with aq = 0. We can rewrite (1.6.2) as
vkq =
a1vk1 + a2vk2 + · · · + aqvkq = 0 (1.6.2)
− a1
vk1
aq
+
− a2
vk2 + · · · + −
aq
aq−1
vkq− 1
aq
.
Since the vectors vk1 , vk2 , . . . , vkq− 1 are in Vkq−1 (since kq−1 < kq, we have kq−1 ≤ kq −1
and hence all vectors in Vkq− 1 are also in Vkq−1), this identity shows that vkq is in Vkq−1,
contradicting the fact that vkq is a leading vector. This shows that all coefficients
a1, a2, . . . , ap must be zeroes.
To prove part (b) of the theorem, take any non-leading vector vj from the list and
let vk1, vk2, . . . , vks be all leading vectors preceding vj. Then we have ks < j < ks+ 1.
Now, for all i with ks < i < ks+ 1, we have Vi = Vks . This is because vi is not a leading
vector and vks+1 is the first leading vector after vks . In particular, Vj = Vks and
hence vj is in Vks . We can repeat the argument for proving Corollary 1.4.2 to prove that
vk1 , vk2 , . . . , vks form a basis of Vks . Now part (b) of Theorem 1.4.1 is clear.
10

EXERCISE SET II. 1.
Review Questions. What is the big deal about linear independence? Can I explain this
concept to my parents and convince them I get my money’s worth for paying tuition fee
to learn important things like this? (Admittedly, this is hard!) Do I understand perfectly
the meaning of each of the following concept?
linear dependence (independence), linear combination, span, spanning set
Given a list of vectors, what are the leading vectors? Why are leading vectors linearly
independent and any vector in this list can be uniquely written as a linear of them? How
do I use a matrix to find the leading vectors and the expression of any vector in the list as
a linear combination of the leading vectors?
Drills
1. In each of the following cases, show that the given set S of vectors in V is linearly
independent:
(a) V = R 3 , S = {(1, 1, 0), (1, 2, 0)}.
(b) V = C 3 , S = {(1, 1, i), (1, i, 1), (i, 1, 1)}.
(c) V = P2, S = {p(x), p ′ (x), p ′′ (x)}, where p(x) = 1 + x + x 2 .
(d) V = P2, S = {p(x), p(x + 1), p(x + 2)}, where p(x) = x2 .
(e) V = M2,2, S = {A, A2
1 −1
}, where A = .
0 −2
2. In each of the following cases, show that the given set S of vectors are linearly dependent:
(a) V = R 3 , S = {(1, 1, 0), (1, 2, 0), (88, 89, 0)}.
(b) V = C 3 , S = {(1, i, 1), (i, 1, i), (1, 0, 1)}.
(c) V = P2, S = {p(x), p(x + 1), p(x + 2), p(x + 3)}, where p(x) = x 2 .
(d) V = M2,2, S = {A, A 2 , A 3 }, where
A =
1 −1
.
0 −2
3. Suppose that u, v, w are linearly independent vectors in a complex vector space V .
Show that
(a) u + v, u − v are linearly independent.
(b) u + v, u + w, v + w are linearly independent.
11

(c) u + iv, iu + v are linearly independent.
4. True or False:
(a) A set S consisting of a single vector, say S = {v}, is always linearly independent.
(b) If two vectors are linearly dependent, then one of them is a scalar multiple of the
other.
(c) If two vectors are linearly dependent, then each of them is a scalar multiple of
the other.
(d) If v1, v2, v3 are linearly dependent, then so are v1, v2.
(e) A set of vectors is linearly independent if and only if none of them is in the span
of the rest of them.
(f) A set of three vectors is linearly independent if each pair of them is a linearly
independent set.
5. Prove the following two statements:
(a) 2×2 matrices A1, A2, A3 (considered as vectors in M2,2) are linearly independent
if BA1, BA2, BA3 are linearly independent, where B is some 2 × 2 matrix.
(b) Polynomials p1(x), p2(x), p3(x) (considered as vectors in P) are linearly independent,
if their derivatives p ′ 1 (x), p′ 2 (x), p′ 3 (x) are linearly independent.
6. In each of the following cases, find the leading vectors of the given list L of vectors
in a linear space V and express the other vectors in the list as linear combinations of
the leading vectors.
(a) V = P2, L = x + 1, 2x + 2, x − 1, 2x, x 2 − 2x, x 2 + 5x + 1
(b) V = C 2 , L = ( (1 + i, 1 − i), (i, 1), (1, i), (1, 2) )
(c) V = C 3 , L = ( C1, C2, C3, C4, C5, C6 ), assuming that the matrix
C = [C1 C2 C3 C4 C5 C6]
has the following reduced row echelon form:
⎡
1
R = ⎣ 0
2
0
0
1
0
0
5
2
⎤
3
4 ⎦
0 0 0 1 6 7
7. verify that subspaces V1, V2, V3 in Example 1.3.1 are invariant for both operators
D and Ta.
12

Exercises
1. Let T be a linear map from V to W and let v1, v2, . . . , vr be vectors in V . Show
that, if T v1, T v2, . . . , T vr are linearly independent, then so are v1, v2, . . . , vr. Give
an example to show that the converse of this statement is false.
2. Show that vectors v1, v2, . . . , vr are linearly independent if and only if v1 = 0 and,
for each k with 2 ≤ k ≤ r, vk is not in the linear span of v1, v2, . . . , vk−1.
3. (a) Let T be a linear operator on a vector space V and let v be a vector in V . Suppose
that n is a positive integer such that T n v = 0 and T n+ 1 v = 0. Show that the vectors
v, T v, T 2 v, . . . , T n v are linearly independent. (b) Use this result to show that
if p(x) is a polynomial of degree n, then the polynomials obtained by consecutive
differentiation p(x), p ′ (x), p ′′ (x), . . . , p (n) (x) are linearly independent.
4. Let P be the 4 × 4 matrix given by
⎡
0 1 0
⎤
0
⎢ 0
P = ⎣
0
0
0
1
0
0 ⎥
⎦ .
1
1 0 0 0
Show that I, P, P 2 , P 3 are linearly independent.
5. (a) Give an example of three linearly dependent vectors v1, v2, v3 in an appropriate
vector space such that every pair of them form a linearly independent set. (b) Give
an example of two linear operators S and T on an appropriate vector space V such
that, as vectors in L (V ), S and T are linearly independent, but, for every v in V , the
vectors Sv and T v are linearly dependent! Hint: Let S = MA and T = MB with
A =
1 0
0 0
and B =
1 1
.
0 0
6. Suppose that H is a subspace of V spanned by the set SH ≡ {h1, h2, . . . , hr} and
K be a subspace of V spanned by SK ≡ {k1, k2, . . . , ks}. Show that the set S ≡
{h1, h2, . . . , hr, k1, k2, . . . , ks} is linearly independent if and only if both sets SH and
SK are linearly independent and H ∩ K = {0}, that is, the zero vector is the only
vector in both H and K.
7. Prove that if H is a subspace of a finite dimensional vector space V , then there is a
subspace K of V such that H + K = V and H ∩ K = {0}. (Hint: Take a basis of H
and extend it to a basis of V .)
13

§2. Dimension
2.1. The main purpose of the present subsection is to establish the theorem stated
below. The key role played by this theorem in linear algebra is to justify the definition of
the dimension of a vector space.
Theorem 2.1.1. Any two bases of a vector space have the same number of vectors.
Let V be a finite dimensional vector space. Take any basis in V ; (we can do this because
V has a basis according to Corollary 1.4.3 in the previous section. This theorem tells us
that the number of vectors in it does not depend on which basis we pick. So we can define
the dimension of V to be the number of vectors in any basis of V , denoted by dim V .
We provide two proofs of this theorem. The first proof given here is based on the
material about leading vectors described in the last section. Let us take two bases of V ,
say E consisting of vectors e1, e2, . . . , em and F consisting of f1, f2, . . . , fn. Here of
course m and n are number of vectors in the bases E and F respectively. We need to prove
m = n. Without loss of generality, let us assume n ≤ m. We use F as a coordinate
system to convert vectors in E into column vectors in F n . Let
Ck = [ek]F (1 ≤ k ≤ m) and C = [C1 C2 · · · Cm].
Notice that C is an n × m matrix, with n ≤ m. Since the vectors e1, e2, . . . , em are
linearly independent, all of them are leading vectors. This fact carries over to the column
vectors of C: all of C1, C2, . . . , Cm are leading vectors. Hence the reduced row echelon
form of C must be ⎡
1 0
⎤
0
⎢ 0
⎢
⎣
1
. ..
0 ⎥
⎦
0 0 1
which is necessarily a square matrix. So C is a square matrix as well. Thus m = n.
Example 2.1.2. Now we determine the dimensions of familiar spaces. First, the
dimension of F n is n, since the standard basis given as follows has exactly n vectors:
e1 = (1, 0, . . . , 0, 0), e2 = (0, 1, . . . , 0, 0), . . . , en = (0, 0, . . . , 0, 1).
The linear space Mmn(F) of all m × n matrices over F can be identified with F mn (as
long as the linear structure is concerned and hence its dimension is mn. Next, the space
Pn of all polynomials of degree not exceeding n is of dimension n + 1, since its standard
basis consisting of
1, x, x 2 , . . . , x n−1 , x n
14

has n + 1 elements. Now we determine the dimension of the space L (V, W ) of all linear
mappings from V into W , where dim V = n and dim W = m. Take a basis E in V and F
in W , Then each linear map T ∈ L (V, W ) corresponds to the m × n matrix [T ] E F and
vice versa. Thus the dimension of L (V, W ) is the same as the dimension of the space of
all m × n matrices, which is clear mn. Finally, th dual V ′ of a vector space is the special
case of L (V, W ) with W = F and hence the dimension of V ′ is 1 × n = n. In summary,
dim F n = n, dim Mmn = mn, dim Pn = n + 1,
dim L (V, W ) = (dim V )(dim W ), dim V ′ = dim V.
Example 2.1.3. Determine dimension of the subspace M of C 3 consisting of vectors
of the form (x, x + iy, 2y), where x, y are arbitrary complex numbers.
Solution. We can rewrite (x, x+iy, 2y) as x(1, 1, 0)+y(0, i, 2). Let b1 = (1, 1, 0),
b2 = y(0, i, 2). Then b1, b2 span M. Also, we can easily check that b1, b2 are linearly
independent. Hence b1, b2 form a basis of M. Therefore dim M = 2.
Example 2.1.4. In Example 1.2.4 from the last section, we took n + 1 distinct
complex numbers c1, c2, . . . , cn+ 1 and consider n+1 vectors vk = (c k 1 , ck 2 , . . . , ck n , ck n+ 1 )
(k = 0, 1, 2, . . . , n) in C n+ 1 . We showed that these vectors are linearly independent. Now
the number of vectors here is the same as the dimension of the space C n+ 1 , namely n + 1.
From the following corollary we see that these vectors form a basis of C n+ 1 .
Corollary 2.1.1. If E is a linearly independent set of m vectors in a linear space
V with dim V = n, then m ≤ n. If, furthermore, m = n, then E forms a basis of V .
Indeed, by Corollary 1.4.4 from the last section, we know that E can be extended to a
basis F of V . Since dim V = n, F is a set of n vectors. Now the corollary is clear.
Corollary 2.1.2. If E is a spanning set of m vectors in a linear space V and with
dim V = n, then m ≤ n. If, furthermore, m = n, then E forms a basis of V .
To see this, we use Corollary 1.4.2 from the last section to infer that E contains a basis
of V , say B. Since dim V = n, there are n vectors in B. The rest of argument is clear.
2.2. Now we give a second proof of Theorem 2.1.1 in a more direct way, not relying on
the material about leading vectors in the previous section and the fact that every matrix
can be row reduced to a reduced row echelon form. However, basic skill in handling the
summation symbol is required and this proof can be skipped or postponed. §2 of
15

Chapter III will be devoted to the basic skill of handling the summation symbol. You may
return to the second proof after studying that section.
Again let E and F be two bases in V , where E consists of vectors e1, e2, . . . , em and
F consists of f1, f2, . . . , fn. Now each vector in E can be written as a linear combination
of vectors in F and vice versa, say
ej = n
For each ℓ with 1 ≤ ℓ ≤ n, we have
k= 1 pjkfk and fk = m
j= 1 qkjej.
fℓ =
j qℓjej =
j qℓj
k pjkfk =
j
=
k j qℓjpjkfk =
k j qℓjpjk
fk.
k qℓjpjkfk
Since f1, f2, . . . , fn form a basis, the coefficient of fk must be the same for each k, equal
to 1 for k = ℓ and zero otherwise. Thus we have
j qℓjpjk
1,
= δℓk ≡
0,
if k = ℓ;
otherwise.
(2.2.1)
(Here, δℓk given above is the so-called Kronecker’s delta.) We can rewrite (2.2.1) in
matrix form as QP = In, where In is the n × n identity matrix and
⎡
⎤ ⎡
⎤
p11 · · · · · · p1m
q11 · · · · · · q1n
⎢
P =
.
⎣ .
. ⎥ ⎢ .
. ⎥
.
. ⎦ , Q = ⎣ .
. ⎦ .
pn1 · · · · · · pnm
qm1 · · · · · · qmn
By reversing the roles of p’s and q’s, we know that (2.2.1) holds if p and q are switched,
giving us P Q = Im.
♠ Aside: Merely P Q = In alone is not enough to guarantee that Q is the inverse of P
(unless you also know that both P and Q are square matrices) One could have AB = In
without the invertibility of A and B. For example, if
A =
⎡ ⎤
1 0
1 0 0
, B = ⎣ 0 1 ⎦ ,
0 1 0
9 9
then AB = I2, but A, B are not invertible because they are not square matrices. ♠
Consider the sum S of all pkjqjk (= qjkpkj), where k runs from 1 to n and j runs
from 1 to m. We can perform the addition in two different ways: one way is letting k run
16

first, the other letting j run first. According to what we have above, (in particular, the
identities (2.1.1)):
S = m
S = n
n
j= 1 k= 1 pkjqjk = m
j= 1
m
k= 1 j= 1 qjkpkj = n
k= 1
1 = m,
1 = n.
Therefore m = n. The second proof of Theorem 2.1.1 is complete.
2.3. Next is a basic theorem about “dimension counting” for a linear map.
Theorem 2.3.1. If T : V → W is a linear transformation from a finite dimensional
vector space V into another W over the same field, then we have
dim ker T + dim T (V ) = dim V.
We call dim ker T (the dimension of the kernel of T ) the nullity of T and we call dim T (V )
(the dimension of the range of T ) the rank of T . So the above theorem says:
nullity + rank = dimension of the domain V
♠ Aside: The above identity has nothing to do with the dimension of W . To understand
Theorem 2.3.1, we think of V as the space of vectors to start with and its size is measured
by dim V . The subspace ker T consists of those v in V nullified (or killed) by T : T v = 0.
The range of T is imagined to be the collection of those vectors who survive and land on
a new place W . So the identity of this theorem says: the size of those who get killed plus
the size of survivors is equal to the total size at the beginning in V . ♠
The proof of this theorem involves several straightforward steps of “unwinding” definitions.
For convenience, let us put n = dim V , k = dim ker T and r = dim T (V ), keeping
in mind that we have to prove k + r = n. That k = dim ker T means that there are k
vectors in ker T , say v1, v2, . . . , vk, which form a basis of ker T . Note that vj being in
ker T (1 ≤ j ≤ k) means T vj = 0. Let us look at another identity: r = dim T (V ). This
means there are r vectors in T (V ), say w1, w2, . . . , wr, which form a basis of T (V ). As
these vectors are in the range T (V ), there are vectors u1, u2, . . . , ur such that
T u1 = w1, T u2 = w2, . . . . . . , T ur = wr.
Let B = {v1, . . . , vk, u1, , . . . , ur}. First we show the linear independence of B. Assume
a1v1 + · · · + akvk + b1u1 + · · · + brur = 0. (2.3.1)
17

(Don’t forget that the theorem we are proving now mainly concerns a linear transformation,
namely T . Naturally we apply T to both sides of the identity and see what happens.) Apply
T to (2.3.1) above and write down T (a1v1 + · · · + akvk + b1u1 + · · · + brur) = T 0. Because
T is linear, we can rewrite this as
a1T v1 + · · · + akT vk + b1T u1 + · · · + brT ur = 0.
Recall what happens when we apply T to these v’s and u’s: T vj = 0, T uk = wk. So
b1w1 + b2w2 + · · · + brwr = 0. (2.3.2)
But w1, w2, . . . , wr form a basis of T (V ). In particular, they are linearly independent. So
(2.3.2) entails b1 = b2 = · · · = br = 0. Now go back to (2.3.1). It becomes
a1v1 + a2v2 + · · · + akvk = 0.
Since v1, v2, . . . , vk are linearly independent, (because they form a basis of ker T ), we have
a1 = a2 = · · · = ak = 0. Hence B ≡ {v1, . . . , vk, u1, . . . , ur} are linearly independent.
Next we prove that B spans V . To this end, take an arbitrary vector v in V . Our goal
is to write v as a linear combination of vectors from B. Applying T to v, we get T v, a vector
in the range T (V ). In T (V ) we have already picked a basis, namely {w1, w2, . . . , wr},
which is waiting to be used. Thus T v as a linear combination of them, say T (v) =
β1w1 + β2w2 + · · · + βrwr. Recall that w1 = T u1, w2 = T u2 etc. So
This tells us T z = 0, where
T v = β1T u1 + · · · + βrT ur = T (β1u1 + · · · + βrur).
z = v − (β1u1 + · · · + βrur). (2.3.3)
In other words, z is in ker T . Now, in ker T there is a basis of vectors waiting for us, namely
v1, v2, . . . , vk. Hence z is a linear combination of these basis vectors, say
z = α1v1 + α2v2 + · · · + αkvk. (2.3.4)
Combining (2.3.3) and (2.3.4), we end up with v = α1v1 + · · · + αkvk + β1u1 + · · · + βrur,
which is exactly what we want.
Theorem 2.3.2. If M and N are subspaces of a (finite dimensional) vector space V ,
then
dim(M + N) + dim(M ∩ N) = dim M + dim N.
18

Here, M + N stands for the subspace consisting of vectors which can be written in the
form u + v, where u is in M and v is in N, and M ∩ N, called the intersection of M and
N, stands for the subspace consisting of vectors in both M and N. Thus
M + N = {u + v: u ∈ M and v ∈ N} M ∩ N = {w ∈ V : w ∈ M and w ∈ N}.
The proof of this Theorem can be done by using similar argument as the previous theorem.
It is left as an exercise for the reader. (See Exercise 13.)
2.4. Now we draw some consequences of Theorem 2.3.1. Given vector spaces V
and W , assume there is an invertible linear mapping T from V to W (Recall that
“T is invertible” means that there is a linear map S from W to V such that ST = IV
and T S = IW .) The invertibility of T tells us that ker T = {0} and T (V ) = W .
Thus the identity dim ker T + dim T (V ) = dim V becomes dim W = dim V and hence
dim V = dim W . We have shown
Corollary 2.4.1. If there is an invertible linear mapping from V to W , then V
and W has the same dimension.
♠ Remark: There is a subject in mathematics called category theory (which is considered
as “abstract general nonsense” by some people) that puts every branch of mathematics
under its umbrella. According to that theory (perhaps “philosophy” is a more appropriate
word here) every branch of mathematics deals with a family of “objects” and “mophisms”
between objects. Invertible morphisms are called isomorphisms. If there is an isomorphisms
between two objects, say X and Y , then we say that X and Y are isomorphic.
Two isomorphic objects are indistinguishable in their behavior and often considered as the
same. In linear algebra, the objects are finite dimensional vector spaces and morphisms are
linear mappings between them. The above corollary tells us that isomorphic vector spaces
have the same dimension. The converse of this is also true: if V and W are vector spaces
over F having the same dimension, say dim V = dim W = n, then they are isomorphic.
Indeed, according to the last paragraph of subsection 2.6 in Chapter I, both V and W are
isomorphic to F n and hence they are isomorphic to each other. Thus, dimension is the
only thing we need to tell if two vector spaces are “the same” or “different”. Comparing
to other branches of mathematics, objects in linear algebra are considered to be “easy”. ♠
Next, let T be a linear operator on a finite dimensional space, (that is, T is a linear
map from V into V itself). Assume that ker T is the zero space, that is, ker T = {0}.
The identity dim ker T + dim T (V ) = dim V becomes dim T (V ) = dim V . Now T (V ) is
a subspace of V , and it has the same dimension as V . So T (V ) must coincide with V .
This shows that all vectors in V are in the range of T , in other words, T is surjective.
19

So, for each v, there is a vector u in V such that T u = v. Here u is uniquely
determined by v. To see this, assume there is another vector w such that T w = v.
Then T (u − w) = T u − T w = v − v = 0. This shows that u − w is in ker T . The
assumption, ker T = {0} tells us that u − w = 0 and hence u = w. Now we can define
the inverse T −1 of T by putting T −1 v = u if T u = v holds. We have proved
Corollary 2.4.2. If T is a linear operator on a finite dimensional space V with
ker T = {0}, then T is invertible.
The argument for proving the above corollary also shows
Corollary 2.4.3. If T is a linear mapping on a finite dimensional space V into
itself and if the homogeneous equation T x = 0 has no nontrivial solution, then, for any
vector b in V , the equation T x = b has a unique solution.
Corollary 2.4.4. If M is an m × n matrix with m < n, then the homogeneous
equation Ax = 0 has nontrivial solutions. (Notice that m is the number of equations and
n is the number of unknowns in the system of equations in vector form Ax = 0.)
Proof. Define the linear map T : C n → C m by putting T x = Ax (in other
words, T = MA). Then dim ker T + dim T (C n ) = dim C n = n. On the other hand,
since T (C n ) is a subspace of C m and dim C m = m, we have dim T (C n ) ≤ m. Thus
dim ker T = dim C n − dim T (C n ) ≥ n − m > 0. Hence ker T = {0}. Any nonzero vector
in ker T gives a nontrivial solution to Ax = 0.
Let A = [A1 A2 · · · An] be an m × n matrix. As usual, we study the homogeneous
equation Ax = 0 by introducing the linear map T : C n → C m . A solution of this equation
is just a vector in the kernel of T . So the dimension of the solution space is the nullity of
T , which is also the number of parameters needed in writing down the general solution.
The rank of A is defined to be the rank of T , which is the dimension of the range T (V )
of V . From then identity
⎢
T x = Ax = [A1 A2 · · · An] ⎢
⎣
⎡
x1
x2
.
xn
⎤
⎥
⎦ = A1x1 + A2x2 + · · · + Anxn
we see that the range of T is just the column space of A, that is, the linear span of
the column vectors A1, A2, . . . , An of the matrix A. The leading vectors of this list of
column vectors is a basis of the column space of A. The number of the leading vectors is
the rank of A. Recall that leading vectors can be identified by the echelon form of A.
20

2.5. Consider the following general problem: given a polynomial p(x) of degree d,
how can we find a formula for
Sn = p(0) + p(1) + p(2) + · · · + p(n) ≡ n
k= 1
p(k) (2.5.1)
which is valid for all n? Well, suppose that we know ahead of time that there is a polynomial
q(x) of degree d + 1 such that q(x + 1) − q(x) = p(x), then from (2.5.1) we have
Sn = (q(1) − q(0)) + (q(2) − q(1)) + (q(3) − q(2)) + · · · + (q(n + 1) − q(n))
= (−q(0) + q(1)) + (−q(1) + q(2)) + (−q(2) + q(3)) + · · · + (−q(n) + q(n + 1))
= q(n + 1) − q(0)
by canceling the neighboring terms. So the required formula is Sn = q(n + 1) − q(n), or
Sn = Q(n), where Q(x) = q(x + 1) − q(0) is a polynomial also of degree d + 1. Once we
know the existence of such q(x), we can find it by solving a system of linear equations,
as shown in the following examples.
Example 2.5.1. Find a formula for the sum Sn = 1 + 2 + 3 + · · · + n.
Solution. Certainly we know this formula from an elementary math course. We apply
the method described here for practice. Here we take p(x) = x, which is a polynomial
of degree 1. So q(x) is a polynomial of degree 2. Let us write q(x) = ax 2 + bx + c.
Then q(x + 1) − q(x) = a(x + 1) 2 + b(x + 1) + c − (ax 2 + bx + c) = 2ax + a + b. So
q(x + 1) − q(x) = p(x) gives 2ax + a + b = x. Comparing both sides leads to the system
2a = 1, a + b = 0. The solution to this system is a = 1/2, b = −1/2. The constant term c
is arbitrary and we set it to zero: c = 0. Hence q(x) = (x 2 − x)/2. Therefore the required
formula is
as we expect.
Sn = q(n + 1) − q(0) = (n + 1)2 − (n + 1)
2
− 0 =
n(n + 1)
2
Example 2.5.2. Find a formula for the sum Sn = 1 2 + 2 2 + 3 2 + · · · + n 2 .
Solution. We use a slightly different method. Take p(x) = x 2 . Motivated from our
discussion above, we try to find a polynomial Q(x) of degree 3 such that Sn = Q(n).
Write Q(x) = ax 3 + bx 2 + cx + d. Then Q(0) = 0, Q(1) = 1, Q(2) = 1 2 + 2 2 = 5 and
Q(3) = 1 2 + 2 2 + 3 2 = 14 gives the following system of equations
d = 0, a + b + c + d = 1, 8a + 4b + 2c + d = 5 27a + 9b + 3c + d = 14.
You only need little patience to find the solution: a = 1/3, b = 1/2, c = 1/6, d = 0. So
Q(x) = x 3 /3 + x 2 /2 + x/6. Thus
Sn = Q(n) = n3
3
+ n2
2
+ n
6
21
= n(n + 1)(2n + 1)
6

which is the require formula.
Now we have the following situation: given p(x), once we known that there is q(x)
such that q(x + 1) − q(x) = p(x), then we can find q(x). The question is, how do we
know such q(x) exists? Now Theorem 2.3.1 comes to help. Denote by Pd the space of all
polynomials of degree not exceeding d. Notice that if q(x) is in Pd+ 1, then the degree
of q(x + 1) − q(x) will be 1 less than that of q(x) (because the highest power terms in
q(x + 1) and q(x) are canceled out) and hence q(x + 1) − q(x) belongs to Pd. Thus we can
define a mapping from Pd+ 1 to Pd by putting T (q(x)) = q(x + 1) − q(x). Now we apply
dim ker T + dim T (V ) = dim V with V = Pd+ 1. As we know, dim V = dim Pd+ 1 = d + 2.
We have to find out dim ker T . Suppose q(x) belongs to ker T . Then q(x + 1) − q(x) = 0,
or q(x) = q(x + 1) (for all x). In particular, q(0) = q(1) = q(2) = q(3) = · · · . This
tells us that all positive integers are roots of q(x) − q(0). But a polynomial cannot
have infinitely many roots unless it is the zero polynomial. Hence q(x) − q(0) = 0, or
q(x) = q(0), that is, q(x) is a constant polynomial. We have shown that ker T is the
space of constant polynomials. Hence dim ker T = 1. Now dim ker T + dim T (V ) = dim V
becomes 1 + dim T (V ) = d + 2, or dim T (V ) = d + 1. Here T (V ) is a subspace of Pd with
dim T (V ) = dim Pd = d + 1 and hence T (V ) = Pd. In particular p(x) ∈ Pd is in the range
of T , showing that there exists q(x) in Pd+ 1 with q(x + 1) − q(x) = p(x).
2.6. Consider a variation of the general problem studied in the last subsection: given
a polynomial p(x) of degree d, and a constant a, how can we find a formula for
Sn = p(0) + p(1)a + p(2)a 2 + · · · + p(n)a n ≡ n
k= 1 p(k)ak
(2.6.1)
which is valid for all n? Certainly, when a = 1, we return to the problem which is dealt
with in the last subsection. So we assume a = 1.
We look for an expression f(x) such that f(k + 1) − f(k) = p(k)a k so that we can
use a “telescoping sum” as in the previous subsection to obtain Sn = f(n + 1) − f(0). Let
us try f(k) = q(k)a k , where q(x) is some polynomial. Now
f(k + 1) − f(k) = q(k + 1)a k+ 1 − q(k)a k = (aq(k + 1) − q(k))a k .
So f(k + 1) − f(k) = p(k)a k becomes (aq(k + 1) − q(k))a k = p(k)a k . Naturally we ask
if there is a polynomial q(x) such that aq(x + 1) − q(x) = p(x). Again, the question is
whether such q(x) exists. Once we know it does exist, we get the license to look for it!
Define a linear operator T on Pd by putting T (q(x)) = aq(x + 1) − q(x). By
Corollary 2.4.3, we know that, in order to show that T (q(x)) = p(x) has a solution (here
p(x) is given and q(x) is unknown), it suffices to show that ker T is the zero space. Let
22

q(x) be in ker T . Then aq(x + 1) = q(x). We claim that q(x) is the zero polynomial.
Suppose the contrary that q(x) = 0. Notice that q(x) cannot be a (nonzero) constant,
due the assumption a = 1. Let m be the degree of q(x). Then q(x + 1) − q(x) is of
degree m − 1 or less. Now we write aq(x + 1) − q(x) = a(q(x + 1) − q(x)) + (1 − a)q(x).
The degree of a(q(x + 1) − q(x)) is m − 1 or less. Since 1 − a = 0, (1 − a)q(x) and q(x)
have the same degree, namely m. Thus the sum a(q(x + 1) − q(x)) + (1 − a)q(x) is a
polynomial of degree m. This is impossible because it is equal to aq(x + 1) − q(x), which
is assumed to be the zero polynomial. So ker T is necessarily the zero space. We conclude:
for a = 1, given any polynomial p(x), there is a polynomial q(x) of the same degree such
that aq(x + 1) − q(x) = p(x).
Example 2.6.1. Again we start with a simple one with the answer well known: find
a formula for Sn = 1 + a + a 2 + · · · + a n . In this case p(x) is the constant polynomial
1 (a polynomial of degree zero) and hence q(x) should also be a constant polynomial,
say q(x) = c. Now aq(x + 1) − q(x) = p(x) becomes ac − c = 1 which gives c = 1/(a − 1).
Thus f(k) = q(k)a k = a k /(a − 1) and the required formula is
which is well known.
Sn = f(n + 1) − f(0) =
1
an+ 1
−
a − 1 a − 1 = an+ 1 − 1
a − 1
Example 2.6.2. Find a formula for Sn = a + 2a 2 + 3a 3 + · · · + na n .
Solution. In the present case we have p(x) = x, which is of degree 1. We look for a
polynomial q(x) of degree 1 such that aq(x + 1) − q(x) = p(x), say q(x) = cx + d. Now
aq(x+1)−q(x) = p(x) becomes a(c(x+1)+d)−(cx+d) = x, or (ac−c)x+ac+ad−d = x,
which leads to the system of equations ac − c = 1, ac + ad − d = 0, with a is given and c, d
are unknowns. Solving this system, we obtain c = 1/(a − 1) and d = −a/(a − 1) 2 . Thus
q(x) =
So we have
(a − 1)x − a
(a − 1) 2
which is the required formula.
(x − 1)a − x
=
(a − 1) 2 and hence f(k) = q(k)a k = (k − 1)ak+ 1 − kak (a − 1) 2
Sn = f(n + 1) − f(0) = nan+ 2 − (n + 1)a n+ 1 + a
(a − 1) 2
23
.

EXERCISE SET II. 2.
Review Questions. Can I give all technical aspects in defining the dimension of a vector
space? Can I make each of the following vague statements precise and prove it?
“Spanning sets have more (at worse equal number of) vectors than independent set.”
“A spanning set can be trimmed down to a basis.”
“An independent set can be extended to a basis.”
“An independent set is a basis of its span.”
Can I explain the identity “rank + nullity = dimension” in detail? Do I know how to
prove this identity?
Drills
1. Give the dimension of each of the following vector spaces over R.
(a) R 4 , (b) P5, (c) M3,4, (d) FX with X = {1, 2, 3, 4}.
2. What is the dimension of C when it is considered as a vector space over R? What
about C n , also considered as a vector space over R?
3. True or False:
(a) A linear homogeneous system of 100 equations with 99 unknowns (or variables)
must have a nontrivial solution.
(b) A linear homogeneous system of 100 equations with 200 unknowns must have a
nontrivial solution.
(c) 100 vectors in a subspace spanned by 50 vectors must be linearly dependent.
(d) If a subspace M is spanned by 50 linearly independent vectors, then the dimension
of M is at most 50.
(e) If vectors v0, v1, v2, . . . , v50 are linearly independent, then the dimension of the
subspace M spanned by them is 50.
(f) If v1, v2 and v3 span a 2-dimensional vector space V , then one of the following
three sets is a basis of V : S1 = {v2, v3}, S2 = {v1, v3}, S3 = {v1, v2}.
(g) If v1, v2, v3, . . . , v99 are linearly independent vectors in a vector space V with
dim V = 100, then we can pick a vector v100 in V such that v1, v2, . . . , v99, v100
form a basis of V .
24

(h) If S is a set of 123 vectors spanning a vector space V with dim V = 100, then we
can delete 23 vectors from S such that the remaining vectors form a basis of V .
(i) If dim V = 100 and v1, . . . , v123 span V , then v1, . . . , v100 form a basis of V .
4. Answer the following questions:
(a) For two subspaces H and K of a vector space V , if the dimensions of H ∩ K, H
and H + K are 2, 4 and 5 respectively, what is the dimension of K?
(b) If M and N are subspaces of P10 with dim M = 7, dim N = 8 and dim(M ∩N) =
5, is M + N = P10?
(c) Describe linear operators of rank zero.
(d) If the kernel of a linear transformation T defined on P4 consists of polynomials
of the form a + bx + (a + b + c)x 2 + cx 3 + ax 4 , where a, b, c are arbitrary scalars,
what is the rank of T ?
5. Find the rank and the nullity of each of the following linear transformations. In each
case, verify the identity “nullity + rank = dimension of the domain”.
(a) T1 : F 2 → F 3 sending (x1, x2) to (0, x1, x2).
(b) T2 : F 3 → F 3 sending (x1, x2, x3) to (0, x1, x2).
(c) Sω : R 3 → R 3 given by Sω(x) = ω × x, where ω is a fixed nonzero vector in R 3 .
(d) D : Pn → Pn sending p(x) ∈ Pn to its derivative p ′ (x).
(e) M : P3 → P5, sending p(x) to x 2 p(x).
(f) E : P2 → P2, sending p(x) to 1
2 (p(x) + p(−x)).
(g) Q : P2 → P2, sending p(x) to 1
2 (p(x) − p(−x)).
1
(h) δ : M2,2 → M2,2, given by δ(X) = AX − XA, where A =
0
0
2
1
(i) Φ : M2,2 → M2,2 given by Φ(X) = BXC, where B =
1
0 1
, C =
0 0
1
.
0
6. True or false (S and T are linear operators on a finite dimensional vector space V of
dimension at least 2):
(a) If ST = T S and if v ∈ ker(T ), then S(v) ∈ ker(T ).
(b) T and T 2 always have the same kernel.
(c) If T and T 2 have the same rank, then they have the same range.
(d) If T and T 2 have the same rank, then they have the same kernel.
(e) The identity rank(S + T )=rank(S)+rank(T ) − rank(ST ) holds.
25
.

(f) If S 2 = O, then rank(S) ≤ nullity of S.
7. We are given matrix A and its reduced row echelon form B as follows:
⎡
2 4i 7i 3 2
⎤
0
⎡
1 2i 3i 1 1
⎤
0
⎢ 1
A = ⎣
0
2i
i
3i
5i
1
2i
2
−2
0 ⎥ ⎢ 0
⎦ =⇒ B = ⎣
0
0
1
0
5
1
2
−i
2i
2i
0 ⎥
⎦ .
0
1 2i 4i 2 0 0
0 0 0 0 0 0
(a) Write down a basis for the range of MA.
(b) Write down a basis for ker(MA).
(c) Write u = (1, 3i, 6i, −1 + 2i, 3, 0) as a linear combination of the row vectors of B.
(d) What is the rank and the nullity of MA?
8. For each linear map, find a basis of its kernel and a basis of its range.
(a) T : C3 → C2
−1 i 1 + i
given by T = MA, where A =
.
i 1 1 − i
(b) S : P3 → P2 given by T (p(x)) = xp ′′ (x) − 2p ′ (x) + p(1).
(c) Φ : M2,2 → M2,2 given by Φ(X) = AX − XA, where A =
1 3
.
0 5
9. Find a polynomial p(x) of degree 4 such that p(x + 1) − p(x) = x(x + 1)(x + 2). Then
use your result to give a formula for the sum n k= 0k(k + 1)(k + 2).
Exercises
1. Give a careful proof of the following statement: a set of vectors in V is a basis of V
if and only if it is linearly independent and it spans V .
2. Prove that if vectors v1, v2, . . . , v50 are linearly independent and are linear combinations
of vectors w1, w2, . . . , w50, then w1, w2, . . . , w50 are linearly independent.
3. Let T be a linear operator defined on a 2-dimensional real vector space such that
T 2 = −I. Show that there is a basis B of V such that [T ]B =
0 −1
1 0
4. Let T be a linear transformation from one vector space V into another W . Define a
new linear tranformation T2 : V × V → W by putting T2(x, y) = T x + T y. Suppose
that the rank of T is r and the dimension of V is n. What is the nullity of T2? Hint:
dim(V × V ) = 2n; T and T2 have the same range.
5. Let T be a linear transformation from a finite dimensional vector space V to another
W . Define a linear transformation T (2) : V → V × V by putting T (2) x = (T x, T x).
Show that T and T (2) have the same rank.
26
.

6. Suppose that T is a linear operator on a 3-dimensional space V satisfying T 2 = O.
Show that the rank of T is at most 1.
7*. Let S and T be linear operators on a finite dimensional vector space V . Show that
(a) rank(S + T ) ≤ rank(S)+ rank(T ).
(b) rank(ST ) ≤ the minimum of rank(S) and rank(T ).
8*. Let S and T be linear operators on a finite dimensional vector space satisfying ST S =
S (i.e. T is a generalized inverse of S.) Show that ST , T S and S have the same rank.
Hint: 7(b) above.
9. Show that a nonzero m × n matrix A is rank one if and only if A can be written as
⎡
a1b1 a1b2 · · ·
⎤
a1bn
⎢ a2b1
⎢
A = ⎢
⎣
a2b2 · · · a2bn ⎥
⎦ ,
amb1 amb2 · · · ambn
for some scalars a1, a2, . . . , am, b1, b2, . . . , bn. Hint : If the rank of A is 1, then the
induced linear transformation MA : F n → F m is a rank one operator and hence its
range is spanned by a single vector in R m , say [a1 a2 · · · am]. It is known that the
columns of A span the range of MA.)
10*. Prove that, for each (a1, a2, a3, a4, a5), there is a unique polynomial p(x) of degree 4
such that p(0) = a1, p ′ (0) = a2, p ′ (0) = a3, p(1) = a4 and p ′ (1) = a5. (Hint: Notice
that, if p(0) = 0, p ′ (0) = 0 and p ′ (0) = 0, then x 3 is a factor of p(x).)
11. Let a be a nonzero constant. (a) Check that if q(x) is in Pn, then (x+a)q(x+1)−xq(x)
is also in Pn. (b) Prove that, given p(x) in Pn, there exists q(x) in Pn such that
(x + a)q(x + 1) − xq(x) = p(x). (Hint: consider the linear map Tn : Pn → Pn
defined by T (q(x)) = (x + a)q(x + 1) − xq(x).)
12. (a) Check that if q(x) is in Pn, then xq(x+1)−(x+1)q(x) is also in Pn. (b) Prove that
there exists a polynomial that cannot be written in the form xq(x + 1) − (x + 1)q(x),
where q(x) is in Pn.
13*. Prove Theorem 2.3.2 as follows. Take a basis w1, w2, . . . , wr in M ∩ N. Extend it
to a basis w1, . . . , wr, u1, u2, . . . , us of M, and a basis w1, . . . , wr v1, v2, . . . , vt
in N. Verify that the vectors w1, w2, . . . , wr u1, u2, . . . , us, v1, v2, . . . , vt form
a basis of M + N.
14*. (Hard problem.) Let p(x) be a polynomial of degree n. Prove that, considered as
vectors in Pn, the polynomials p(x), p(x + 1), . . . , p(x + n) are linearly independent.
27

Appendix A*: quotient spaces
Appendices for Chapter II
Let V be a vector space, not necessarily finite dimensional. For subsets S and T in V ,
and a scalar a, we define S + T (aS respectively) to be the set of vectors which can be
expressed in the form u + v (au respectively) with u in S and v in T , that is,
S + T = {u + v | u ∈ S and v ∈ T }, aS = {au | u ∈ S}.
When S consists of a single point u, we write u + T for S + T . Let M be a linear subspace
of V . We call a subset of V of the form u + M (where u is some vector in V ) a coset
(of M). Notice that u in generally is not uniquely determined. In fact, it is easy to check
that two cosets u + M and v + M are equal if and only if u − v is in M. In case V is
a 2–dimensional space represented by a plane with a point representing the zero vector,
called the origen, then a 1–dimensional subspace M is a line through the origin of this
plane and the cosets of M are lines parallel the to line representing M.
It is easy to show that if A and B are cosets of M, then so are A + B and aA, where
a is any scalar. In fact, when A = u + M and B = v + M, we have A + B = u + v + M
and aA = au + M. Furthermore, with addition and scalar multiplication of cosets defined
in this manner, the collection of all cosets form a vector space. This vector space of all
cosets of M in V is called the quotient space of V over M, denoted by V/M:
V/M = {u + M| u ∈ V }.
There is a natural linear map Q from V to the quotient space V/M given by Qu = u+M.
The map Q : V → V/M is called the quotient map. It is easy to check that the range
of Q is V/M and the kernel of Q is M. Hence it follows from Theorem 2.3.1 of the present
chapter that, when V is finite dimensional,
dim V/M = dim V − dim M.
Now suppose that N is another subspace of V (assumed to be finite dimensional). Let R
be the restriction of Q to N. Thus R: N → V/M is the linear map given by Ru = u+M
for all u in N. Then we can check that the kernel of R is M ∩ N and the range of R is
(N + M)/M. So, by Theorem 2.3.1 again, we get
dim N − dim(M ∩ N) = dim (M + N)/M = dim(M + N) − dim N.
28

Hence dim(M + N) + dim M ∩ N = dim M + dim N. Thus we have given a conceptual
proof of Theorem 2.3.2.
Let T : V → W be a linear map between vector spaces. Let M = ker T , the kernel
of T and let Q : V → V/M be the quotient map. Then there is a unique isomorphism
˜T : V/M → T (V ) such that T = j ˜ T Q, where j : T (V ) → W is the inclusion map for
the subspace T (V ) of W . This fact roughly says that a linear map is essentially a quotient
map, followed by an injection. (It is analogous to the first isomorphism theorem in group
theory or ring theory.)
Appendix B*: Duality
Recall that the dual V ′ of a vector space V over F is the space of all linear functionals
on V : V ′ = L (V, F). The dual of V ′ , naturally denoted by V ′′ , is called the bidual of V .
There is a natural map J : V → V ′′ defined by (Ju)(φ) = φ(u) for all u ∈ V and φ ∈ V ′′ .
Now we prove: when V is finite dimensional, J is an isomorphism between V and V ′′ .
As we know, dim V ′ = dim V . Applying this identity to V ′ , we have dim V ′′ = dim V ′ .
Consequently dim V ′′ = dim V . So it is enough to show that ker J = {0}. To this end,
we show that, given a nonzero vector u in V , Ju = 0, there is, there exists φ ∈ V ′ such
that (Ju)(φ) = φ(u) = 0. Since u = 0, there is a basis of vectors b1, . . . , bn in V such
that b1 = u. Now define a linear functional φ by putting φ(x1b1 + · · · + xnbn) = x1 for
any vector x = x1b1 + · · · + xnbn in V . Then φ(u) = 1 = 0. Now we can conclude that
J : V → V ′′ is an isomorphism. Using this natural isomorphism J, we can treat V ′′ and V
as the same and hence V as the dual space of V ′ .
Let M be a subspace of V . The annihilator M o of M is defined as follows:
M o = {φ ∈ V ′ | φ(x) = 0 for all x ∈ M}.
Let φ be an element in M o . Then φ(v + M) = φ(v) for all v ∈ V and hence φ can
be treated as a linear functional on V/M. Conversely, if ψ is a linear functional on V/M,
then ψ ◦ Q is a linear functional on V belonging to M o . This shows that the dual of V/M
is naturally isomorphic to M o : (V/M) ′ ∼ = M o . By a more sophisticated argument, we can
show M ′ ∼ = V ′ /M o .
Appendix C*: Projective Space
Let V be a vector space over F of dimension n + 1. For any positive integer k with
k ≤ n, denote by Gk(V ) the set of all k–dimensional subspaces of V ; (G here stands
29

for Grassmann). The n-dimensional projective space modeled on V , denoted by P [V ]
here, is G1(V ), the set consisting of all 1-dimensional subspaces of V . Thus a point in
P [V ] is just a 1–dimensional subspace. For a nonzero vector v in V , denote by [v] the
1–dimensional subspace spanned by v, which represents a point in P [V ]. Thus we may
write
P [V ] = G1(V ) = {[v] | v ∈ V, v = 0}.
Notice that two points [u] and [v] in P [V ] coincide if and only if u = av for some scalar
a = 0. Given M ∈ G2(V ) (that is, M is a 2–dimensional subspace of V ), denote by [M]
the set of all points [v] in P [V ] with v ∈ M:
[M] = {[v] ∈ P [V ] | v ∈ M}.
In other words, [M] is the set of all 1–dimensional subspaces contained in M. A set of
the form [M] with M ∈ G2(V ) is called a (projective) line. Now the statement “given
two distinct points in a projective space, there is a unique line passing them” in projective
geometry can be rephrased as “given two distinct 1–dimensional subspaces, there is a
unique 2–dimensional subspace containing both of them” in linear algebra, which is more
or less transparent. In the same way we define a (projective) plane in P [V ] to be [M] =
{[v] ∈ P [V ] | v ∈ M}, where M ∈ G3(V ).
When V = F n+ 1 , we write P n (F) or simply P n for P [V ]. For a nonzero vector
x = (x0, x1, . . . , xn) in F n+ 1 , we write [x0 : x1 : x2 : · · · : xn] for [x]; (the numbers
x0, x1, . . . , xn are called the homogeneous coordinates of the point). Notice that, for any
scalar a = 0, [ax0 : ax1 : · · · : axn] = [x0 : x1 : · · · : xn]. We identify a point
(x1, x2, . . . , xn) in F n with the point [1 : x1 : · · · : xn] in the projective space P n , called
an ordinary point. Notice that, when x0 = 0, [x0 : x1 : · · · : xn] is an ordinary point
because it can be rewritten as [1 : x1/x0 : · · · : xn/x0]. Thus a non–ordinary point is of
the form [0 : x1 : · · · : xn]; it is called a point at infinity.
Now we give a closer look at the case n = 2 and F = R. In this case P 2 is called the
real projective plane. Let us consider a line
ax + by = c (C1)
in F 2 . Write x = x1/x0 and y = x2/x0. Then we have a(x1/x0) + b(x2/x0) = c, or
a0x0 + a1x1 + a2x2 = 0, (C2)
where a0 = −c, a1 = a, a2 = b are not simultaneously zero. When [x0 : x1 : x2] is an
ordinary point on the line given by (C2) in the projective plane, it also represents a point
30

(x, y) with x = x1/x0, y = x2/x0 on the line given by (C2) in the Euclidean plane R 2 .
Take any point (p, q) on the line (C1), that is, ap + bq = c. We can put (C1) in parametric
form as x = p + bt, y = q − at. The point (x, y) = (p + bt, q − at) can be identified
with the point [1 : p + bt : p − at] in the projective plane, which can be rewritten as
[t −1 : t −1 p + b : t −1 p − a] (assuming t = 0). Letting t → ∞, we get a point at infinity,
namely [0 : b : −a]. Clearly x0 = 0, x1 = b, x2 = −a satisfy (C2). Thus [0 : b : −a] is the
point on the line (C2) at infinity. Notice that the vector (b, −a) is parallel to the line
(C1). So it points in the direction where the point at infinity on the line can be found.
When a0 = 1, a1 = a2 = 0, (C2) becomes x0 = 0. We call x0 = 0 the equation for the
line at infinity. The point [0 : b : −a] is the intersection of the line (C2) and the line at
infinity. Using the fact that two distinct 2–dimensional subspaces in R 3 intersections in a
1–dimensional subspace, we deduce that two distinct lines in the projective plane intersect
at a unique point. From the projective geometric point of view, two parallel Euclidean line
intersect at infinity.
The process of adding points at infinity to extend the line (C1) to (C2) is called the
completion of (C1). In fact, this process of completion by adding points at infinity works
more generally for algebraic curves. The tangent lines to completed curves at points of
infinity give us asymptotic lines of curves. We give an example to show this
Example C1. Consider the quadratic curve x 2 −2xy −3y 2 +4x+3y +1 = 0. Letting
x = x1/x0 and y = x2/x0, we get x 2 1 −2x1x2 −3x 2 2 +4x1x0 +3x2x0 +x 2 0 = 0. If [0 : x1 : x2]
is a point at infinity of the curve, we have x 2 1 −2x1x2 −3x 2 2 = 0, or (x1 −3x2)(x1 +x2) = 0.
Hence there are two points at infinity, namely [0 : 3 : 1] and [0 : −1 : 1]. In general, the
tangent line at a point p = [p0 : p1 : p2] on an algebraic curve f(x0, x1, x2) = 0 in the
projective plane is given by the recipe
f0(p)x0 + f1(p)x1 + f2(p)x2 = 0, where fk = ∂f/∂xk; k = 0, 1, 2.
At present, f0 = 4x1 + 3x2 + 2x0, f1 = 2x1 − 2x2 + 4x0, f2 = −2x1 − 6x2 + 3x0. At point
[0 : 3 : 1], putting x0 = 0, x1 = 3 and x2 = 1, we get we have f0 = 15, f1 = 4, f2 = −12.
So the tangent line at this point becomes 15x0 + 4x1 − 12x2 = 0. So the corresponding
asymptote is 4x − 12y + 15 = 0. We can find the tangent line to the other point [0 : −1 : 1]
at infinity in the same way to obtain the second asymptote.
Projective spaces provide important examples in geometry and topology. When F is
a finite field, Pn(F) is crucial for combinatorial design.
31