PDF of Lecture Notes - School of Mathematical Sciences

2. STATISTICAL INFERENCE
Hence the CRLB is achieved only if U is a linear function of T with probability 1
(correlation equals 1):
U = aT + b a, b constants =⇒ can depend on θ but not x
i.e.
∂l
∂θ
Integrating wrt θ, we obtain:
where A ′ (θ) = a(θ), B ′ (θ) = b(θ).
= U(θ; x) = a(θ)T (x) + b(θ) for all x.
log f(x; θ) = l(θ; x) = A(θ)T (x) + B(θ) + h(x),
Hence,
f(x; θ) = exp{A(θ)T (x) + B(θ) + h(x)}.
Finally to ensure that E(T ) = θ, recall that E{U(θ; X)} = 0 and observe that in this
case,
E{U(θ; X)} = E[A ′ (θ)T (X) + B ′ (θ)]
= A ′ (θ)E[T (X)] + B ′ (θ) = 0
=⇒ E[T (X)] = −B′ (θ)
A ′ (θ) .
Hence, in order that T (X) be unbiased for θ, we must have −B′ (θ)
A ′ (θ)
= θ.
Definition. 2.2.3
A probability density function/probability function is said to be a single parameter
exponential family if it has the form:
f(x; θ) = exp{A(θ)t(x) + B(θ) + h(x)}
for all x ∈ D ∈ R, where the D does not depend on θ.
If x 1 , . . . , x n is a random sample from an exponential family, the joint PDF/prob functions
becomes
n∏
f(x; θ) = exp{A(θ)t(x i ) + B(θ) + h(x i )}
i=1
{
= exp A(θ)
}
n∑
n∑
t(x i ) + B(θ) + h(x i ) .
i=1
i=1
88