Iterative Methods for Sparse Linear Systems Second Edition

6 CHAPTER 1. BACKGROUND IN LINEAR ALGEBRA
1.4 Vector Inner Products and Norms
An inner product on a (complex) vector space X is any mapping s from X × X into
C,
x ∈ X, y ∈ X → s(x, y) ∈ C,
which satisfies the following conditions:
1. s(x, y) is linear with respect to x, i.e.,
s(λ1x1 + λ2x2, y) = λ1s(x1, y) + λ2s(x2, y), ∀ x1, x2 ∈ X, ∀ λ1, λ2 ∈ C.
2. s(x, y) is Hermitian, i.e.,
3. s(x, y) is positive definite, i.e.,
s(y, x) = s(x, y), ∀ x, y ∈ X.
s(x, x) > 0, ∀ x = 0.
Note that (2) implies that s(x, x) is real and therefore, (3) adds the constraint that
s(x, x) must also be positive for any nonzero x. For any x and y,
s(x,0) = s(x,0.y) = 0.s(x, y) = 0.
Similarly, s(0, y) = 0 for any y. Hence, s(0, y) = s(x,0) = 0 for any x and y. In
particular the condition (3) can be rewritten as
s(x, x) ≥ 0 and s(x, x) = 0 iff x = 0,
as can be readily shown. A useful relation satisfied by any inner product is the socalled
Cauchy-Schwartz inequality:
|s(x, y)| 2 ≤ s(x, x) s(y, y). (1.2)
The proof of this inequality begins by expanding s(x − λy, x − λy) using the properties
of s,
s(x − λy, x − λy) = s(x, x) − ¯ λs(x, y) − λs(y, x) + |λ| 2 s(y, y).
If y = 0 then the inequality is trivially satisfied. Assume that y = 0 and take
λ = s(x, y)/s(y, y). Then, from the above equality, s(x − λy, x − λy) ≥ 0 shows
that
|s(x, y)|2
0 ≤ s(x − λy, x − λy) = s(x, x) − 2
= s(x, x) −
s(y, y)
|s(x, y)|2
s(y, y) ,
+ |s(x, y)|2
s(y, y)