PDF of Lecture Notes - School of Mathematical Sciences

2. STATISTICAL INFERENCE
X 1 , . . . , X n i.i.d. Exp(λ). We saw previously that ˜λ X = 1¯X .
Suppose Y i = X 2 i (which is invertible for X i > 0). To obtain ˜λ Y , observe E(Y ) =
E(X 2 ) = 2 λ 2 √
=⇒ ˜λ 2n
Y = ∑ ≠ 1¯X . X
2
i
2.3.2 Maximum Likelihood Estimation
Consider a statistical problem with log-likelihood function, l(θ; x).
Definition. The maximum likelihood estimate ˆθ is the solution to the problem max l(θ; x)
θ∈Θ
i.e. ˆθ = arg max l(θ; x).
θ∈Θ
Remark
In practice, maximum likelihood estimates are obtained by solving the score equation
Example
∂l
∂θ
= U(θ; x) = 0
If X 1 , X 2 , . . . , X n are i.i.d. geometric-θ RV’s, find ˆθ.
Solution:
{ n
}
∏
l(θ; x) = log θ(1 − θ) x i−1
i=1
n∑
n∑
= log θ + (x i − 1) log(1 − θ)
i=1
i=1
= n{log θ + (¯x − 1) log(1 − θ)}
∴ U(θ; x) = ∂l { 1
∂θ = n θ − ¯x − 1
1 − θ
= n
( 1 − θ¯x
θ(1 − θ)
)
.
}
97