First Semester in Numerical Analysis with Julia, 2020a

CHAPTER 5. APPROXIMATION THEORY 200
normal equations as a matrix equation has certain drawbacks:
• The integrals ∫ b
a xi+j dx =(b i+j+1 − a i+j+1 ) /(i + j +1)in the coefficient matrix gives
rise to matrix equation that is prone to roundoff error.
• There is no easy way to go from P n (x) to P n+1 (x) (which we might want to do if we
were not satisfied by the approximation provided by P n ).
There is a better approach to solve the discrete and continuous least squares problem using
the orthogonal polynomials we encountered in Gaussian quadrature. Both discrete and
continuous least squares problem tries to find a polynomial P n (x) = ∑ n
j=0 a jx j that satisfies
some properties. Notice how the polynomial is written in terms of the monomial basis
functions x j and recall how these basis functions caused numerical difficulties in interpolation.
That was the reason we discussed different basis functions like Lagrange and Newton for the
interpolation problem. So the idea is to write P n (x) in terms of some other basis functions
P n (x) =
n∑
a j φ j (x)
j=0
which would then update the normal equations for continuous least squares (5.11) as
n∑
∫ b
a j φ j (x)φ k (x)dx =
j=0
a
∫ b
a
f(x)φ k (x)dx
for k =0, 1, ..., n. The normal equations for the discrete least squares (5.1) gets a similar
update:
(
n∑ m
)
∑
m∑
a j φ j (x i )φ k (x i ) = y i φ k (x i ).
j=0
i=1
Going forward, the crucial observation is that the integral of the product of two functions
∫
φj (x)φ k (x)dx, or the summation of the product of two functions evaluated at some discrete
points, ∑ φ j (x i )φ k (x i ), can be viewed as an inner product 〈φ j ,φ k 〉 of two vectors in a suitably
defined vector space. And when the functions (vectors) φ j are orthogonal, the inner product
〈φ j ,φ k 〉 is 0 if j ≠ k, which makes the normal equations trivial to solve. We will discuss
details in the next section.
i=1
5.3 Orthogonal polynomials and least squares
Our discussion in this section will mostly center around the continuous least squares problem,
however, the discrete problem can be approached similarly. Consider the sets C 0 [a, b], the