Theory of Statistics - George Mason University

272 3 Basic Statistical Theory
Example 3.21 Risk functions for estimators of the parameter in a
binomial distribution
Suppose we have an observation X from a binomial distribution with parameters
n and π. The PDF (wrt the counting measure) is
n
pX(x) = π
x
x (1 − π) n−x I{0,1,...,n}(x).
We wish to estimate π.
The MLE of π is
T(X) = X/n. (3.107)
We also see that this is an unbiased estimator. Under squared-error loss, the
risk is
RT(π) = E (X/n − π) 2 = π(1 − π)/n,
which, of course, is just variance.
Let us consider a randomized estimator, for some 0 ≤ α = 1,
T with probability 1 − α
δα(X) =
1/2 with probability α
(3.108)
This is a type of shrunken estimator. The motivation to move T toward 1/2 is
that the maximum of the risk of T occurs at 1/2. By increasing the probability
of selecting that value the risk at that point will be reduced, and so perhaps
this will reduces the risk in some overall way.
Under squared-error loss, the risk of δα(X) is
Rδα(π) = (1 − α)E (X/n − π) 2 + αE (1/2 − π) 2
= (1 − α)π(1 − π)/n + α(1/2 − π) 2 .
The mass point has a spreading effect and the risk dips smoothly The risk
of δα(X) also has a maximum at π = 1/2, but it is (1 − α)/4n, compared to
RT(1/2) = 1/4n.
We see that for α = 1/(n + 1) the risk is constant with respect to π;
therefore δ1/(n+1)(X) is a minimax estimator wrt squared-error loss.
Risk functions are shown in Figure 3.1 for T(X) and for δ.05(X) and
δ1/(n+1)(X). Notice that neither δα(X) nor T(X) dominates the other.
3.3.5 Summary and Review
We have discussed five general approaches to statistical inference, and we have
identified certain desirable properties that a method of inference may have.
A first objective in mathematical statistics is to characterize optimal properties
of statistical methods. The setting for statistical inference includes the
distribution families that are assumed a priori, the objectives of the statistical
Theory of Statistics c○2000–2013 James E. Gentle