Linear Algebra, Theory And Applications, 2012a

296 INNER PRODUCT SPACES
∑
(Av k ,w l )(v p ,v k )(w l ,w r )
k,l
=(Av p ,w r ) − ∑ k,l
(Av k ,w l ) δ pk δ rl
=(Av p ,w r ) − (Av p ,w r )=0.
Letting A − ∑ k,l (Av k,w l ) w l ⊗ v k = B, this shows that Bv p =0sincew r is an arbitrary
element of the basis for Y. Since v p is an arbitrary element of the basis for X, it follows
B = 0 as hoped. This has shown {w j ⊗ v i : i =1, ··· ,n, j =1, ··· ,m} spans L (X, Y ) .
It only remains to verify the w j ⊗ v i are linearly independent. Suppose then that
∑
c ij w j ⊗ v i =0
Then do both sides to v s . By definition this gives
i,j
0= ∑ i,j
c ij w j (v s ,v i )= ∑ i,j
c ij w j δ si = ∑ j
c sj w j
Now the vectors {w 1 , ··· ,w m } are independent because it is an orthonormal set and so the
above requires c sj = 0 for each j. Since s was arbitrary, this shows the linear transformations,
{w j ⊗ v i } form a linearly independent set.
Note this shows the dimension of L (X, Y )=nm. The theorem is also of enormous
importance because it shows you can always consider an arbitrary linear transformation as
a sum of rank one transformations whose properties are easily understood. The following
theorem is also of great interest.
Theorem 12.4.5 Let A = ∑ i,j c ijw i ⊗ v j ∈L(X, Y ) where as before, the vectors, {w i } are
an orthonormal basis for Y and the vectors, {v j } are an orthonormal basis for X. Then if
the matrix of A has entries M ij , it follows that M ij = c ij .
Also
Proof: Recall
Av i ≡ ∑ k
M ki w k
Av i = ∑ k,j
c kj w k ⊗ v j (v i )= ∑ k,j
c kj w k (v i ,v j )
= ∑ k,j
c kj w k δ ij = ∑ k
c ki w k
Therefore,
∑
M ki w k = ∑ c ki w k
k
k
and so M ki = c ki for all k. This happens for each i.
12.5 Least Squares
A common problem in experimental work is to find a straight line which approximates as
well as possible a collection of points in the plane {(x i ,y i )} p i=1
. The usual way of dealing
with these problems is by the method of least squares and it turns out that all these sorts
of approximation problems can be reduced to Ax = b where the problem is to find the best
x for solving this equation even when there is no solution.