STA 36-786: Bayesian Theoretical Statistics I

STA 36-786: Bayesian Theoretical Statistics I
Assignment 2: Spring 2013
Due: Thursday, February 7 at 10:30 a.m.
Show all your work to obtain full/partial credit. You are not to consult outside sources other
than your class notes/slides/reference books for this assignment (except for the instructor
or TA). No late assignments will be accepted. Please follow the instructions for writing up
solutions given out on 1.12.13. Start each problem on a new page.
1. Suppose a < x < b. Consider the notation I (a,b) (x), where I denotes the indicator
function. We define I (a,b) (x) to be the following:
Let
I (a,b) (x) =
{
1 if a < x < b,
0 otherwise.
X|θ ∼ Uniform(0, θ)
θ ∼ Pareto(α, β),
where p(θ) = αβα
θ α+1 I (β,∞)(θ). Calculate the posterior distribution of θ|x.
2. Let X 1 , . . . , X n be iid Poisson(θ) variables, where θ ∈ (0, ∞). Let L(θ, δ) = (θ − δ) 2 /θ.
Assuming the prior,
g(θ) = exp{−θα}αβ θ β−1
I
Γ(β) [θ>0] ,
where α > 0 and β > 0 are given. Show that the Bayes estimator of θ is given by
⎧∑
⎨ i X i + β − 1
if ∑ i
h(X) = n + α
X i + β − 1 > 0
⎩
0 if otherwise.
3. Suppose X | p ∼ Bin(n, p) and that p ∼ Beta(a, b).
(a) Show that the marginal distribution of X is the beta-binomial distribution with
mass function
( ) n Γ(a + b) Γ(x + a)Γ(n + b − x)
m(x) =
.
x Γ(a)Γ(b) Γ(n + a + b)

(b) Show that the mean and variance of the beta-binomial is given by EX =
na
( ) ( ) ( )
a + b
a b a + b + n
and VX = n
.
a + b a + b a + b + 1
Hint: For part(b): Use the formulas for iterated expectation and iterated variance.
4. DasGupta (1994) presents an identity relating the Bayes risk to bias, which illustrates
that a small bias can help achieve a small Bayes risk. Let X ∼ f(x|θ) and θ ∼ π(θ).
The Bayes estimator under squared error loss is ˆδ = E(θ|X). Show that the Bayes
risk of ˆδ can be written
∫ ∫
∫
r(π, ˆδ) = [θ − ˆδ(X)] 2 f(x|θ)π(θ)dx dθ = − θ b(θ)π(θ) dθ
Θ X
Θ
where b(θ) = E[ˆδ|θ] − θ is the bias of ˆδ.
5. Suppose that
Using the HB model above,
X|θ ∼ f(x|θ)
θ|λ ∼ π(θ|λ)
λ ∼ π(λ).
(a) prove that E[θ|x] = E[ E[θ|x, λ] ].
(b) prove that V [θ|x] = E[ V [θ|x, λ] ] + V [ E[θ|x, λ] ].
Remark: when proving (a) and (b) above, you may show this two ways, either
by integrals in which say what you are integrating over or you may simply just
use expectations (and in this case specifying what you are taking an expectation
over).
6. Albert and Gupta (1985) investigate theory and application of the hierarchical model
X i |θ i
ind
∼ Bin(n, θ i ), i = 1, . . . , p
θ i |η ∼ Beta(kη, k(1 − η)), k known
η ∼ Uniform(0, 1).
(a) Show that
and
E(θ i |x) =
V (θ i |x) = E[θ i|x](1 − E[θ i |x])
n + k + 1
Hint: You should show along the way that
n x i
n + k n + k
n + k E(η|x)
+
k 2 V (η|x)
(n + k)(n + k + 1) .
V (θ i |x) = x i(n + k − x i ) + E(η|x)k(n + k − 2x i ) − k 2 E(η 2 |x)
(n + k) 2 (n + k + 1)
+ k2 V (η|x)
(n + k) 2 .
General Remark: Note that E(η|x) and V (η|x) are not expressible in a simple
form and hence you can leave them as such.
2

(b) Unconditionally on η, the θ i ’s have conditional covariance
Show this.
Cov(θ i , θ j |x) =
( ) k 2
V (η|x) for i ≠ j.
n + k
(c) Ignoring the prior on η, show how to construct an EB estimator of θ i . Again,
this is not expressible in a simple form. That is, simply derive the marginal
distribution and then explain using software how you would find an estimator
for η. Then give a simple construction for the EB estimator.
3