Student Notes To Accompany MS4214: STATISTICAL INFERENCE

Suppose that a statistical model specifies that the data x has a probability distribu-
tion f(x; θ) depending on a vector of m unknown parameters θ = (θ1, . . . , θm). In this
case the likelihood function is a function of the m parameters θ1, . . . , θm and having
observed the value of x is defined as L(θ) = f(x; θ) with ℓ(θ) = ln L(θ).
The MLE of θ is a value ˆ θ for which L(θ), or equivalently ℓ(θ), attains its maximum
value. For r = 1, . . . , m define Sr(θ) = ∂ℓ/∂θr. Then we can (usually) find the MLE
ˆθ by solving the set of m simultaneous equations Sr(θ) = 0 for r = 1, . . . , m. The
information matrix I(θ) is defined to be the m×m matrix whose (r, s) element is given
by Irs where Irs = −∂ 2 ℓ/∂θr∂θs. The conditions for a value ˆ θ satisfying Sr( ˆ θ) = 0 for
r = 1, . . . , m to be a MLE are that all the eigenvalues of the matrix I( ˆ θ) are positive.
2.6 Newton-Raphsom Optimization
Example 2.11 (Radioactive Scatter). A radioactive source emits particles intermittently
and at various angles. Let X denote the cosine of the angle of emission. The angle
of emission can range from 0 degrees to 180 degrees and so X takes values in [−1, 1].
Assume that X has density
1 + θx
f(x|θ) =
2
for −1 ≤ x ≤ 1 where θ ∈ [−1, 1] is unknown. Suppose the data consist of n indepen-
dently identically distributed measures of X yielding values x1, x2, ..., xn. Here
L(θ) = 1
2 n
n�
(1 + θxi)
i=1
l(θ) = −n ln [2] +
S(θ) =
I(θ) =
n�
i=1
n�
i=1
xi
1 + θxi
n�
ln [1 + θxi]
i=1
x 2 i
(1 + θxi) 2
Since I(θ) > 0 for all θ, the MLE may be found by solving the equation S(θ) = 0. It
is not immediately obvious how to solve this equation.
By Taylor’s Theorem we have
0 = S( ˆ θ)
= S(θ0) + ( ˆ θ − θ0)S ′ (θ0) + ( ˆ θ − θ0) 2 S ′′ (θ0)/2 + ....
31