Student Notes To Accompany MS4214: STATISTICAL INFERENCE

Example 2.3 (Binomial sampling). The number of successes in n Bernoulli trials is a
random variable R taking on values r = 0, 1, . . . , n with probability mass function
� �
n
P (R = r) = θ
r
r (1 − θ) n−r .
This is the exact same sampling scheme as in the previous example except that instead
of observing the sequence y we only observe the total number of successes r. Hence the
likelihood function has the form
LR (θ|r) =
� �
n
θ
r
r (1 − θ) n−r .
The relevant mathematical calculations are as follows:
� �
n
ℓR (θ|r) = ln + r ln(θ) + (n − r) ln(1 − θ)
r
S (θ) = r n − r
+
n 1 − θ
⇒ ˆ θ = r
n
I (θ) = r n − r
+
θ2 (1 − θ) 2 > 0 ∀ θ
E( ˆ θ) = E(r) nθ
=
n n = θ ⇒ ˆ θ unbiased
Var( ˆ θ) = Var(r)
n2 nθ(1 − θ)
=
n2 = θ(1 − θ)
n
E [I (θ)] = E(r) n − E(r) nθ n − nθ
+ = +
θ2 (1 − θ) 2 θ2 (1 − θ) 2
=
n
θ(1 − θ) =
�
Var[ ˆ �−1 θ]
and ˆ θ attains the Cramer-Rao lower bound (CRLB). �
Example 2.4 (Germinating seeds). Suppose 25 seeds were planted and r = 5 seeds
germinated. Then ˆ θ = r/n = 0.2 and Var( ˆ θ) = 0.2 × 0.8/25 = 0.0064. The relative
likelihood is
R1(θ) =
� �5 � �20 θ 1 − θ
.
0.2 0.8
Suppose 100 seeds were planted and r = 20 seeds germinated. Then ˆ θ = r/n = 0.2
but Var( ˆ θ) = 0.2 × 0.8/100 = 0.0016. The relative likelihood is
R2(θ) =
� θ
0.2
� 20 � 1 − θ
0.8
� 80
.
Suppose 25 seeds were planted and it is known only that r ≤ 5 seeds germinated. In
this case the exact number of germinating seeds is unknown. The information about θ
is given by the likelihood function
L (θ) = P (R ≤ 5) =
21
5�
r=0
� �
25
θ
r
r (1 − θ) 25−r .