TUTORIAL 5 SOLUTIONS #8.10.47 The Pareto distribution has ...

TUTORIAL 5 SOLUTIONS
#8.10.47 The Pareto distribution has been
used in economics as a model for a density
function with a slowly decaying tail:
f(x|x 0 ,θ)=θx θ 0 x−θ−1 , x ≥ x 0 ,θ >1.
Assume that x 0 > 0 is given and that
X 1 ,...,X n is an i.i.d. sample.
a. Find the method of moments estimate of
θ.
b. Find the mle of θ.
c. Find the asymptotic variance of the mle.
d. Find a sufficient statistic for θ.
Solution
a. Let µ = E(X 1 ). Then
µ =
∫ ∞
x 0
xθx θ 0 x−θ−1 dx
1

= θx θ 0 ( x−θ+1
−θ +1 )|∞ x 0
= x 0θ
θ − 1 .
Thus θ = µ/(µ − x 0 ) and the method of
moments estimate of θ is
ˆθ =ˆµ 1 /(ˆµ 1 − x 0 )= ¯X/( ¯X − x 0 ).
b. The loglikelihood function is
n∏
l(θ) = log[ θx θ 0 x−θ−1 i
]
Thus
i=1
= n log(θ)+nθ log(x 0 )
n∑
−(θ +1) log(x i ), x ≥ x 0 .
i=1
l ′ (θ) = n θ + n log(x 0) −
n∑
log(x i ).
i=1
2

Solving for l ′ (θ) = 0, the mle of θ is given
by
n
˜θ = ∑ ni=1
log(X i ) − n log(x 0 ) .
c. The asymptotic variance of the mle is
1/[nI(θ)]. Now
I(θ) =−E[ ∂2
∂θ 2 log f(X 1|θ)]
= −E{ ∂2
∂θ 2[log(θ)
+θ log(x 0 ) − (θ + 1) log(X 1 )]}
= 1 θ 2.
Hence the asymptotic variance of the mle ˜θ
is θ 2 /n.
3

d. Observe that the joint pdf of X =
(X 1 ,...,X n )is
n∏
f(x|θ) =
where
i=1
θx θ 0 x−θ−1 i
= θ n x nθ
0 ( n ∏
i=1
= g(t, θ)h(x),
t =
n∏
x i ,
i=1
x i ) −θ−1
g(t, θ) =θ n x nθ
0 t−θ−1 ,
h(x) =1.
By the factorization theorem, T (X) = ∏ n
i=1 X i
is sufficient for θ.
4

#8.10.53 Let X 1 ,...,X n be i.i.d. uniform
on [0,θ].
a. Find the method of moments estimate of
θ and its mean and variance.
b. Find the mle of θ.
c. Find the probability density of the mle,
and calculate its mean and variance. Compare
the variance, the bias, and the mean
squared error to those of the method of
moments estimate.
d. Find a modification of the mle that renders
it unbiased.
Solution
a. Let µ = E(X 1 ). Then
and θ =2µ.
µ = 1 θ
∫ θ
0
xdx = θ 2 ,
5

Thus the method of moments estimate of
θ is
ˆθ =2ˆµ 1 =2¯X.
Also,
E(ˆθ) =2E( ¯X) =2( θ 2 )=θ,
Var(ˆθ) = Var(2 ¯X) = 4 n Var(X 1)= θ2
3n ,
since
Var(X 1 )= 1 ∫ θ
x 2 dx − (EX
θ
1 ) 2 = θ2
12 .
b. The likelihood function is
n∏ 1
lik(θ) =
θ I{x i ≤ θ}
i=1
0
= 1
θ nI{max(x 1,...,x n ) ≤ θ}.
The maximum of lik(θ) occurs at θ = max(x 1 ,
...,x n ) and hence the mle of θ is
˜θ = max(X 1 ,...,X n ).
6

c. Observe that the cdf of ˜θ is given by
F˜θ(x) =P (˜θ ≤ x)
= P (max(X 1 ,...,X n ) ≤ x)
n∏
= P (X i ≤ x)
i=1
=( x θ )n .
The pdf f˜θ(x) is
f˜θ(x) = d
dx F˜θ(x) = nxn−1
θ n ,
whenever 0 ≤ x ≤ θ. Also,
E(˜θ) = 1 ∫ θ
θ n nx n dx =
nθ
0 n +1 ,
bias = E(˜θ) − θ = −
θ
n +1 ,
Var( ˜θ) = 1 ∫ θ
θ n nx n+1 dx − ( nθ
n +1 )2
=
0
nθ 2
(n +1) 2 (n +2) .
7

The MSE of ˜θ is
MSE(˜θ) =Var(˜θ) + Bias 2
=
=
nθ 2
(n +1) 2 (n +2) +(− θ
n +1 )2
2θ 2
(n + 1)(n +2) .
Comparison of ˆθ and ˜θ
Even though ˆθ is an unbiased estimator
of θ while ˜θ is a biased estimator of θ, the
MSE of ˜θ is dramatically smaller (for large
n) than the MSE of ˆθ.
d. The following modification of the mle
makes it unbiased:
θ ∗ (n +1)˜θ
=
n
since
E(θ ∗ )= n +1
n
E(˜θ) =θ.
8

#8.10.57 This problem is concerned with
the estimation of the variance of a normal
distribution with unknown mean from a sample
X 1 ,...,X n of i.i.d. normal random variables.
In answering the following questions,
use the fact that (from Theorem B of Section
6.3)
(n − 1)s 2
∼ χ 2 n−1
σ 2
and that the mean and variance of a chisquare
random variable with r df are r and
2r respectively.
a. Which of the following estimates is unbiased?
s 2 = 1 n∑
(X
n − 1 i − ¯X) 2 ,
i=1
ˆσ 2 = 1 n∑
(X
n i − ¯X) 2 .
i=1
9

. Which of the estimates given in part (a)
has the smaller MSE?
c. For what value of ρ does
n∑
ρ (X i − ¯X) 2
i=1
have the minimal MSE?
Solution
a. Recall from Section 6.3 that (n−1)s 2 /σ 2 ∼
χ 2 n−1 distribution. Hence
(n − 1)s2
E
σ 2 = n − 1
which implies that
E(s 2 )=σ 2 ,
E(ˆσ 2 )= n − 1
n σ2 .
Thus s 2 is an unbiased estimate of σ 2 .
10

. Since (n − 1)s 2 /σ 2 ∼ χ 2 n−1 distribution,
we have
(n − 1)s2
Var(
σ 2 )=2(n − 1).
Thus
Var(s 2 )=
2σ4
n − 1 ,
Var(ˆσ 2 ) = Var( n − 1 2(n −
n s2 1)σ4
)=
n 2 ,
MSE(s 2 ) = Var(s 2 )+[E(s 2 ) − σ 2 ] 2
= 2σ4
n − 1 ,
MSE(ˆσ 2 ) = Var(ˆσ 2 )+[E(ˆσ 2 ) − σ 2 ] 2
=
=
2(n − 1)σ4
n 2 + σ4
n 2
(2n − 1)σ4
n 2 .
11

Consequently we conclude that ˆσ 2 has the
smaller MSE since
c. Let
MSE(ˆσ 2 ) < MSE(s 2 ).
n
ˆσ ρ 2 = ρ ∑
(X i − ¯X) 2 .
i=1
Then ˆσ 2 ρ =(n − 1)ρs2 . As in part (b), we
have
E(ˆσ ρ)=(n 2 − 1)ρσ 2 ,
Var(ˆσ ρ)=(n 2 − 1) 2 ρ 2 Var(s 2 )
=2(n − 1)ρ 2 σ 4 .
12

Finally,
and
MSE(ˆσ ρ)
2
=Var(ˆσ ρ)+[E(ˆσ 2 ρ) 2 − σ 2 ] 2
=2(n − 1)ρ 2 σ 4 +[nρσ 2 − (ρ +1)σ 2 ] 2
= σ 4 (1 + 2ρ − 2nρ − ρ 2 + n 2 ρ 2 ).
d
dρ MSE(ˆσ2 ρ )
= σ 4 (2 − 2n − 2ρ +2n 2 ρ).
Solving for (d/dρ)MSE(ˆσ ρ 2 ) = 0, we obtain
ρ = 1
n +1 .
ˆσ ρ 2 has the smallest MSE when ρ =1/(n +
1).
13

#8.10.60 Let X 1 ,...,X n be an i.i.d. sample
from an exponential distribution with
the density function
f(x|τ) = 1 τ e−x/τ , 0 ≤ x

g. Find the form of an exact confidence interval
for τ.
Solution
a. Writing X =(X 1 ,...,X n ), the loglikelihood
function is
n∏ 1
l(τ) = log
τ e−x i/τ
Also,
i=1
= −n log(τ) − 1 τ
l ′ (τ) =− n τ + 1
τ 2
n ∑
i=1
n∑
x i .
i=1
x i .
Solving for l ′ (τ) = 0, the mle of τ is
ˆτ = ¯X.
15

. We note from Chapter 4.5 of the text
that
S = X 1 + ···+ X n ∼ Γ(n, 1 τ ).
Hence the pdf of ¯X = S/n is
f ¯X(x) =
sn−1
τ n Γ(n) e−s/τ | ds
dx |
= nn x n−1
τ n Γ(n) e−nx/τ , x > 0,
which is the pdf of the Γ(n, n/τ) distribution.
c. and d. Since ¯X ∼ Γ(n, n/τ), we have
E( ¯X) =τ,
Var( ¯X) = τ 2
n .
From the CLT, ( ¯X − τ)/ √ τ 2 /n is approximately
distributed as N(0, 1) for large n.
16

e. The Cramér-Rao lower bound is 1/[nI(τ)]
where
I(τ) =−E[ ∂2
∂τ 2 log(1 τ e−X1/τ )]
= − 1
τ 2 + E(2X 1
τ 3 )
= 1
τ 2,
since E(X 1 ) = τ. This implies that the
Cramér-Rao lower bound is
[nI(τ)] −1 = τ 2 /n.
This lower bound equals the variance of ¯X.
Hence we conclude that there is no other
unbiased estimate of τ with a smaller variance
than ¯X.
17

f. From part (c), we have ( ¯X −τ)/ √ τ 2 /n
is approximately distributed as N(0, 1) for
large n.
Hence an approximate 100(1 − α)% CI for
τ is
¯X ± z 1−α/2
τ
√ n
≈ ¯X ± z 1−α/2
¯X√n ,
or equivalently the set of τ’s satisfying
τ
τ − z 1−α/2 √ ≤ ¯X τ
≤ τ + z n 1−α/2 √ . n
18

g. Note that ¯X has exactly the Γ(n, n/τ)
distribution.
Let G τ (α) denote the 100α percentile of
the Γ(n, n/τ) distribution, i.e.
P ( ¯X ≤ G τ (α)) = α.
Then an exact 100(1 − α)% CI for τ is
given by the set of τ’s satisfying
G τ (α/2) ≤ ¯X ≤ G τ (1 − α/2).
19