PDF of Lecture Notes - School of Mathematical Sciences

2. STATISTICAL INFERENCE
=⇒ MSE T (θ) = Var(T ) =
θ(1 − θ)
.
n
Remark
This example shows that MSE T (θ) must be thought of as a function of θ rather than
just a number.
E.g., Figure 18
Figure 18: MSE T (θ) as a function of θ
Intuitively, a good estimator is one for which MSE is as small as possible. However,
quantifying this idea is complicated, because MSE is a function of θ, not just a number.
See Figure 19.
For this reason, it turns out it is not possible to construct a minimum MSE estimator
in general.
To see why, suppose T ∗ is a minimum MSE estimator for θ. Now consider the estimator
T a = a, where a ∈ R is arbitrary. Then for a = θ, T θ = θ with MSE Tθ (θ) = 0.
Observe MSE Ta (a) = 0; hence if T ∗ exists, then we must have MSE T ∗(a) = 0. As a
is arbitrary, we must have MSE T ∗(θ) = 0 ∀θ, ∈ Θ
=⇒ T ∗ = θ with probability 1.
Therefore we conclude that (excluding trivial situations) no minimum MSE estimator
can exist.
Definition. 2.1.5 The bias of the estimator T is defined by:
b T (θ) = E(T ) − θ.
79