First Semester in Numerical Analysis with Julia, 2020a

CHAPTER 4. NUMERICAL QUADRATURE AND DIFFERENTIATION 175
She tells her classmates about the cancellation of leading digits phenomenon, and
to make her point more convincing, she makes the following experiment: let f(x) =e x ,
and suppose we want to compute f ′ (2) which is e 2 . Arya uses the three-point midpoint
formula above to estimate f ′ (2), for various values of h, and for each case she computes
the absolute value of the difference between the three-point midpoint estimate and the
exact solution e 2 . She tabulates her results in the table below. Clearly, smaller h does
not result in smaller error. In this experiment, h =10 −6 gives the smallest error.
h 10 −4 10 −5 10 −6 10 −7 10 −8
Abs. error 1.2 × 10 −8 1.9 × 10 −10 5.2 × 10 −11 7.5 × 10 −9 2.1 × 10 −8
Theoretical analysis
Numerical differentiation is a numerically unstable problem. To reduce the truncation error,
we need to decrease h, which in turn increases the roundoff error due to cancellation of
significant digits in the function difference calculation. Let e(x) denote the roundoff error in
computing f(x) so that f(x) = ˜f(x)+e(x), where ˜f is the value computed by the computer.
Consider the three-point midpoint formula:
∣ f ′ (x 0 ) − ˜f(x 0 + h) − ˜f(x 0 − h)
2h ∣
=
∣ f ′ (x 0 ) − f(x 0 + h) − e(x 0 + h) − f(x 0 − h)+e(x 0 − h)
2h
∣
=
∣ f ′ (x 0 ) − f(x 0 + h) − f(x 0 − h)
+ e(x 0 − h) − e(x 0 + h)
2h
2h ∣
=
∣ −h2 6 f (3) (ξ)+ e(x 0 − h) − e(x 0 + h)
2h ∣ ≤ h2
6 M + ɛ h
where we assumed |f (3) (ξ)| ≤M and |e(x)| ≤ɛ. To reduce the truncation error h 2 M/6 one
would decrease h, which would then result in an increase in the roundoff error ɛ/h. An
optimal value for h can be found with these assumptions using Calculus: find the value for
h that minimizes the function s(h) = Mh2 + ɛ . Theanswerish = 3√ 3ɛ/M.
6 h
Let’s revisit the table Arya presented where 10 −6 was found to be the optimal value for
h. The calculations were done using Julia, which reports 15 digits when asked for e 2 . Let’s
assume all these digits are correct, and thus let ɛ =10 −16 . Since f (3) (x) =e x and e 2 ≈ 7.4,
let’s take M =7.4. Then
h = 3√ 3ɛ/M = 3√ 3 × 10 −16 /7.4 ≈ 3.4 × 10 −6