Theory of Statistics - George Mason University

390 5 Unbiased Point Estimation
UMVU is closely related to complete sufficiency, which means that it probably
has nice properties (like being able to be identified easily) in exponential
families. One of the most useful facts is the Lehmann-Scheffé theorem.
Theorem 5.1 (Lehmann-Scheffé Theorem)
Let T be a complete sufficient statistic for θ, and suppose T has finite second
moment. If g(θ) is U-estimable, then there is a unique UMVUE of g(θ) of the
form h(T), where h is a Borel function.
The first part of this is just a corollary to the Rao-Blackwell theorem,
Theorem 3.8. The uniqueness comes from the completeness, and of course,
means unique a.e.
The Lehmann-Scheffé theorem may immediately identify a UMVUE.
Example 5.5 UMVUE of Bernoulli parameter
Consider the Bernoulli family of distributions with parameter π. Suppose
we take a random sample X1, . . ., Xn. Now the Bernoulli (or in this case,
the binomial(n, π)) is a complete one-parameter exponential family, and T =
n
i=1 Xi is a complete sufficient statistic for π with expectation nπ. By the
Lehmann-Scheffé theorem, therefore, the unique UMVUE of π is
W =
n
Xi/n. (5.5)
i=1
In Example 3.17, page 265, we showed that the variance of W achieves the
CRLB; hence it must be UMVUE.
The random sample from a Bernoulli distribution is the same as a single
binomial observation, and W is an unbiased estimator of π, as in Example
5.1. We also saw in that example that a constrained random sample from
a Bernoulli distribution is the same as a single negative binomial observation
N, and an unbiased estimator of π in that case is (t − 1)/(N − 1), where t is
the required number of 1’s in the constrained random sample. This estimator
is also UMVU for π (Exercise 5.1).
In the usual definition of this family, π ∈ Π =]0, 1[. Notice that if
n
i=1 Xi = 0 or if n i=1 Xi = n, W /∈ Π. Hence, the UMVUE may not be valid
in the sense of being a legitimate parameter for the probability distribution.
Useful ways for checking that an estimator is UMVU are based on the
following theorem and corollary.
Theorem 5.2
Let P = {Pθ}. Let T be unbiased for g(θ) and have finite second moment.
Then T is a UMVUE for g(θ) iff E(TU) = 0 ∀θ ∈ Θ and ∀U ∋ E(U) =
0, E(U 2 ) < ∞ ∀θ ∈ Θ.
Proof.
First consider “only if”.
Theory of Statistics c○2000–2013 James E. Gentle