PDF of Lecture Notes - School of Mathematical Sciences

2. STATISTICAL INFERENCE
(2) f is s.t. f(x; θ 1 ) = f(x; θ 2 ) for all x =⇒ θ 1 = θ 2 .
We will show (in outline) that if ˆθ n is the MLE (maximum likelihood estimator) based
on X 1 , X 2 , . . . , X n , then:
(1) ˆθ n → θ 0 in probability, i.e., for each ɛ > 0,
lim P (|θ n − θ 0 | > ɛ) = 0.
n→∞
(2)
√
ni(θ0 )(ˆθ n − θ 0 ) −→ D
N(0, 1) as n → ∞
Remark
(Asymptotic Normality).
The practical use of asymptotic normality is that for large n,
( )
1
ˆθ ∼: N θ 0 , .
ni(θ 0 )
Outline of consistency & asymptotic normality for MLE’s:
Consider i.i.d. data X 1 , X 2 , . . . with common PDF/prob. function f(x; θ).
If ˆθ n is the MLE based on X 1 , X 2 , . . . , X n , we will show (in outline) that ˆθ n → θ 0 in
probability as n → ∞, where θ 0 is the true value for θ.
Lemma. 2.3.1
Suppose f is such that f(x; θ 1 ) = f(x; θ 2 ) for all x =⇒ θ 1 = θ 2 . Then l ∗ (θ) =
E{log f(x; θ)} is maximized uniquely by θ = θ 0 .
Proof.
l ∗ (θ) − l ∗ (θ 0 ) =
=
≤
=
=
∫ ∞
−∞
∫ ∞
−∞
∫ ∞
−∞
∫ ∞
−∞
∫ ∞
−∞
(log f(x; θ))f(x; θ 0 )dx −
( f(x; θ)
log
f(x; θ 0 )
)
f(x; θ 0 )dx
( f(x; θ)
f(x; θ 0 ) − 1 )
f(x; θ 0 )dx
(f(x; θ) − f(x; θ 0 ))dx
f(x; θ)dx −
∫ ∞
−∞
∫ ∞
−∞
f(x; θ 0 )dx.
(log f(x; θ 0 ))f(x; θ 0 )dx
102