PDF of Lecture Notes - School of Mathematical Sciences

2. STATISTICAL INFERENCE
To find ˆθ, we solve for θ in 1 − θ¯x = 0
⎡
=⇒ ˆθ = 1¯x
⎢
⎣ =
Elementary Properties
n∑
n =
x i
i=1
# of successes
# of trials
⎤
⎥
⎦
(1) MLE’s are invariant under invertible transformations of the data.
Proof. (Continuous i.i.d. RV’s, strictly increasing/decreasing translation)
Suppose X has PDF f(x; θ) and Y = h(X), where h is strictly monotonic;
=⇒ f Y (y; θ) = f X (h −1 (y); θ)|h −1 (y) ′ | when y = h(x).
Consider data x 1 , x 2 , . . . , x n and the transformed version y 1 , y 2 , . . . , y n :
{ n
}
∏
l Y (θ; y) = log f Y (y i ; θ)
= log
= log
i=1
{
∏ n
}
f X (h −1 (y i ); θ)|h −1 (y i ) ′ |
i=1
n∏
f X (x i ; θ) + log
i=1
n∏
|h −1 (y i ) ′ |
( n
)
∏
= l X (θ; x) + log |h −1 (y i ) ′ | ,
i=1
( n
)
∏
since log |h −1 (y i ) ′ | does not depend on θ, it follows that ˆθ maximizes
i=1
l X (θ; x) iff it maximizes l Y (θ; y).
i=1
(2) If φ = φ(θ) is a 1-1 transformation of θ, then the MLE’s obey the transformation
rule, ˆφ = φ(ˆθ).
Proof. It can be checked that if l φ (φ; x) is the log-likelihood with respect
to φ, then
l θ (θ; x) = l φ (φ(θ); x).
It follows that ˆθ maximizes l θ (θ; x) iff ˆφ = φ(ˆθ) maximizes l φ (φ; x).
98