Theory of Statistics - George Mason University

94 1 Probability Theory
an(g(Xn) − g(bn)) d → Y, (1.193)
where Y ∼ Nk(0, (∇g(b)) T Σ∇g(b)).
One reason limit theorems such as Theorem 1.46 are important is that they
can provide approximations useful in statistical inference. For example, we
often get the convergence of expression (1.191) from the central limit theorem,
and then the convergence of the sequence {g(Xn)} provides a method for
determining approximate confidence sets using the normal distribution, so
long as ∇g(b) = 0. This method in asymptotic inference is called the delta
method, and is illustrated in Example 1.25 below. It is particularly applicable
when the asymptotic distribution is normal.
The Case of ∇g(b) = 0
Suppose ∇g(b) = 0 in equation (1.192). In this case the convergence in distribution
is to a degenerate random variable, which may not be very useful. If,
however, Hg(b) = 0 (where Hg is the Hessian of g), then we can use a second
order the Taylor series expansion and get something useful:
2a 2 n(g(Xn) − g(bn)) d → X T Hg(b)X, (1.194)
where we are using the notation and assuming the conditions of Theorem 1.46.
Note that while an(g(Xn)−g(bn)) may have a degenerate limiting distribution
at 0, a 2 n(g(Xn)−g(bn)) may have a nondegenerate distribution. (Recalling that
limn→∞ an = ∞, we see that this is plausible.) Equation (1.194) allows us also
to get the asymptotic covariance for the pairs of individual elements of Xn.
Use of expression (1.194) is called a second order delta method, and is
illustrated in Example 1.25.
Example 1.25 an asymptotic distribution in a Bernoulli family
Consider the Bernoulli family of distributions with parameter π. The variance
of a random variable distributed as Bernoulli(π) is g(π) = π(1 − π). Now,
suppose X1, X2, . . . iid ∼ Bernoulli(π). Since E(Xn) = π, we may be interested
in the distribution of Tn = g(Xn) = Xn(1 − Xn).
From the central limit theorem (Theorem 1.38),
√ n(Xn − π) → N(0, π(1 − π)), (1.195)
and so if π = 1/2, g ′ (π) = 0, we can use the delta method from expression
(1.192) to get
√ n(Tn − g(π)) → N(0, π(1 − π)(1 − 2π) 2 ). (1.196)
If π = 1/2, g ′ (π) = 0 and this is a degenerate distribution, so we cannot
use the delta method. Let’s use expression (1.194). The Hessian is particularly
simple.
Theory of Statistics c○2000–2013 James E. Gentle