Student Notes To Accompany MS4214: STATISTICAL INFERENCE

When θ ∈ R1 we can define the score function as the first derivative of the log-likelihood
S(θ) = ∂
ln L(θ).
∂θ
The maximum likelihood estimate (MLE) ˆ θ of θ is the solution to the score equation
S(θ) = 0.
At the maximum, the second partial derivative of the log-likelihood is negative, so we
define the curvature at ˆ θ as I( ˆ θ) where
I(θ) = − ∂2
ln L(θ).
∂θ2 We can check that a solution ˆ θ of the equation S(θ) = 0 is actually a maximum by
checking that I( ˆ θ) > 0. A large curvature I( ˆ θ) is associated with a tight or strong
peak, intuitively indicating less uncertainty about θ. In likelihood theory I(θ) is a
key quantity called the observed Fisher information, and I( ˆ θ) is the observed Fisher
information evaluated at the MLE ˆ θ. Although I(θ) is a function, I( ˆ θ) is a scalar.
The likelihood function L(θ|x) supplies an order of preference or plausibility among
possible values of θ based on the observed y. It ranks the plausibility of possible values
of θ by how probable they make the observed y. If P (x|θ = θ1) > P (x|θ = θ2) then
the observed x makes θ = θ1 more plausible than θ = θ2, and consequently from
(2.2), L(θ1|x) > L(θ2|x). The likelihood ratio L(θ1|x)/L(θ2|x) = f(θ1|x)/f(θ2|x) is
a measure of the plausibility of θ1 relative to θ2 based on the observed fact y. The
relative likelihood L(θ1|x)/L(θ2|x) = k means that the observed value x will occur k
times more frequently in repeated samples from the population defined by the value θ1
than from the population defined by θ2. Since only ratios of likelihoods are meaningful,
it is convenient to standardize the likelihood with respect to its maximum. Define the
relative likelihood as R(θ|x) = L(θ|x)/L( ˆ θ|x). The relative likelihood varies between 0
and 1. The MLE ˆ θ is most plausible value of θ in that it makes the observed sample
most probable. The relative likelihood measures the plausibility of any particular value
of θ relative to that of ˆ θ.
When the random variables X1, . . . , Xn are mutually independent we can write the
joint density as
fX(x) =
j=1
where x = (x1, . . . , xn) ′ is a realization of the random vector X = (X1, . . . , Xn) ′ , and
the likelihood function becomes
LX(θ|x) =
n�
n�
j=1
fXj (xj)
fXj (xj|θ).
When the densities fXj (xj) are identical, we unambiguously write f(xj).
19