First Semester in Numerical Analysis with Julia, 2020a

CHAPTER 5. APPROXIMATION THEORY 204
Theorem 92. 1. If φ j (x) is a polynomial of degree j for j =0, 1, ..., n, then φ 0 , ..., φ n are
linearly independent.
2. If φ 0 , ..., φ n are linearly independent in P n , then for any q(x) ∈ P n , there exist unique
constants c 0 , ..., c n such that q(x) = ∑ n
j=0 c jφ j (x).
Exercise 5.3-2: Prove that if {φ 0 ,φ 1 , ..., φ n } is a set of orthogonal functions, then they
must be linearly independent.
We have developed what we need to solve the least squares problem using orthogonal
polynomials. Let’s go back to the problem statement:
Given f ∈ C 0 [a, b], find a polynomial P n (x) ∈ P n that minimizes
E =
∫ b
a
w(x)(f(x) − P n (x)) 2 dx = 〈f(x) − P n (x),f(x) − P n (x)〉
with P n (x) written as a linear combination of orthogonal basis polynomials: P n (x) =
∑ n
j=0 a jφ j (x). In the previous section, we solved this problem using calculus by taking
the partial derivatives of E with respect to a j and setting them equal to zero. Now we will
use linear algebra:
〈
E = f −
n∑
a j φ j ,f −
j=0
〉
n∑
a j φ j = 〈f,f〉−2
j=0
= ‖f‖ 2 − 2
= ‖f‖ 2 − 2
= ‖f‖ 2 −
n∑
a j 〈f,φ j 〉 + ∑ ∑
a i a j 〈φ i ,φ j 〉
j=0
i j
n∑
n∑
a j 〈f,φ j 〉 + a 2 j 〈φ j ,φ j 〉
j=0
n∑
a j 〈f,φ j 〉 +
j=0
n∑ 〈f,φ j 〉 2
+
α j
j=0
n∑
j=0
j=0
n∑
a 2 jα j
j=0
[ 〈f,φj 〉
√
αj
− a j
√
αj
] 2
.
Minimizing this expression with respect to a j is now obvious: simply choose a j = 〈f,φ j〉
[
〈f,φj 〉
√ αj
that the last summation in the above equation, ∑ ]
n
√ 2,
j=0
− a j αj vanishes. Then we
have solved the least squares problem! The polynomial that minimizes the error E is
α j
so
P n (x) =
n∑
j=0
〈f,φ j 〉
α j
φ j (x) (5.13)