Theory of Statistics - George Mason University

674 0 Statistical Mathematics
if the derivatives exist and the series is convergent. (The class of functions
for which this is the case in some region that contains x and a is said to be
analytic over that region; see page 650. An important area of analysis is the
study of analyticity.)
In applications, the series is usually truncated, and we call the series with
k + 1 terms, the kth order Taylor expansion.
For a function of m variables, it is a rather complicated expression:
⎛
∞
f(x1, . . ., xm) = ⎝ 1
m
(xk − ak)
j!
∂
⎞
j
f(x1, . . ., xm) ⎠
∂xk
j=0
k=1
(x1,...,xm)=(a1,...,am)
(0.0.82)
The second order Taylor expansion for a function of an m-vector is the
much simpler expression.
f(x) ≈ f(a) + (x − a) T ∇f(a) + 1
2 (x − a)T Hf(a)(x − a), (0.0.83)
where ∇f(a) is the vector of first derivatives evaluated at a and Hf(a) is the
matrix of second second derivatives (the Hessian) evaluated at a. This is the
basis for Newton’s method in optimization, for example. Taylor expansions
beyond the second order for vectors becomes rather messy (see the expression
on the right side of the convergence expression (1.198) on page 95, for
example).
0.0.9.8 Mean-Value Theorem
Two other useful facts from calculus are Rolle’s theorem and the mean-value
theorem, which we state here without proof. (Proofs are available in most
texts on calculus.)
Theorem 0.0.18 (Rolle’s theorem)
Assume the function f(x) is continuous on [a, b] and differentiable on ]a, b[. If
f(a) = f(b), then there exists a point x0 with a < x0 < b such that f ′ (x0) = 0.
Theorem 0.0.19 (mean-value theorem)
Assume the function f(x) is continuous on [a, b] and differentiable on ]a, b[.
Then there exists a point x0 with a < x0 < b such that
0.0.9.9 Evaluation of Integrals
f(b) − f(a) = (b − a)f ′ (x0).
There are many techniques that are useful in evaluation of a definite integral.
Before attempting to evaluate the integral, we should establish that the
integral is finite. For example, consider the integral
Theory of Statistics c○2000–2013 James E. Gentle
.