PDF of Lecture Notes - School of Mathematical Sciences

2. STATISTICAL INFERENCE
Remark
If x 1 , x 2 , . . . , x n are independent, the log likelihood function can be written as:
l(θ; x) =
n∑
log f i (x i ; θ).
i=1
If x 1 , x 2 , . . . , x n are i.i.d., we have:
l(θ; x) =
n∑
log f(x i ; θ).
i=1
Definition. 2.2.2
Consider a statistical problem with log-likelihood l(θ; x). The score is defined by:
and the Fisher information is
U(θ; x) = ∂l
∂θ
I(θ) = E
( )
− ∂2 l
.
∂θ 2
Remark
For a single observation with PDF f(x, θ), the information is
)
i(θ) = E
(− ∂2
∂θ log f(X, θ) .
2
In the case of x 1 , x 2 , . . . , x n i.i.d. , we have
Theorem. 2.2.1
Under suitable regularity conditions,
I(θ) = ni(θ).
E{U(θ; X)} = 0
and Var{U(θ; X)} = I(θ).
82