PDF of Lecture Notes - School of Mathematical Sciences

2. STATISTICAL INFERENCE
Definition. 2.1.3
A statistic T with property T (x) ∈ Θ ∀x is called an estimator for θ.
Example
For x 1 , x 2 , . . . , x n i.i.d. N(µ, σ 2 ), we have θ = (µ, σ 2 ). The quantity (¯x, s 2 ) is an estimator
for θ, where ¯x = 1 n∑
x i , S 2 = 1 n∑
(x i − ¯x) 2 .
n
n − 1
i=1
There are two important concepts here: the first is that estimators are random
variables; the second is that you need to be able to distinguish between random variables
and their realisations. In particular, an estimate is a realisation of a random
variable.
For example, strictly speaking, x 1 , x 2 , . . . , x n are realisations of random variables
X 1 , X 2 , . . . , X n , and ¯x = 1 n∑
x i is a realisation of ¯X; ¯X is an estimator, and ¯x is an
n
estimate.
i=1
We will find that from now on however, that it is often more convenient, if less rigorous,
to use the same symbol for estimator and estimate. This arises especially in the use of
ˆθ as both estimator and estimate, as we shall see.
An unsatisfactory aspect of Definition 2.1.3 is that it gives no guidance on how to
recognize (or construct) good estimators.
Unless stated otherwise, we will assume that θ is a scalar parameter in the following.
In broad terms, we would like an estimator to be “as close as possible to” θ “with high
probability.
Definition. 2.1.4
The mean squared error of the estimator T of θ is defined by
i=1
MSE T (θ) = E{(T − θ) 2 }.
Example
Suppose X 1 , . . . , X n are i.i.d. Bernoulli θ RV’s, and T = ¯X =‘proportion of successes’.
Since nT ∼ B(n, θ) we have
E(nT ) = nθ, Var(nT ) = nθ(1 − θ)
#
=⇒ E(T ) = θ, Var(T ) =
θ(1 − θ)
n
78