Theory of Statistics - George Mason University

338 4 Bayesian Inference
(x − µ) 2
+ (µ − µ0) 2
σ 2
σ 2 0
= x2
σ2 + µ20 σ2 +
0
σ2 + σ2 0
σ2σ2 µ
0
2 − 2 σ2 µ0 + σ2 0x σ2σ2 µ (4.31)
0
= x2
σ2 + µ20 σ2
+ µ
0
2 − 2 σ2 µ0 + σ2 0x σ2 + σ2 2 2 σ σ0 µ /
0 σ2 + σ2
0
= x2
σ2 + µ20 σ2 −
0
(σ2 µ0 + σ2 0x)2 σ2σ2 0 (σ2 + σ2 0 )
+ µ − σ2 µ0 + σ2 0x
σ2 + σ2 2 2 2 σ σ0 /
0 σ2 + σ2
0
The last quadratic in the expression above corresponds to the exponential in
a normal distribution with a variance of σ2σ2 0/(σ2 +σ2 0), so we adjust the joint
PDF so we can integrate out the µ, leaving
1 σ
fX(x) = √
2πσσ0
2 + σ2 0
σ2σ2
exp −
0
1
2 x
2 σ2 + µ20 σ2 −
0
(σ2 µ0 + σ2 0x) 2
σ2σ2 0 (σ2 + σ2 0 )
.
Combining the exponential in this expression with (4.31), we get the exponential
in the conditional posterior PDF, again ignoring the −1/2 while factoring
out σ2σ2 0/(σ2 + σ2 0), as
µ 2 − 2 σ2 µ0 + σ2 0x
σ2 + σ2 µ +
0
(σ2 µ0 + σ2 0x) 2
(σ2 + σ2 0 )2
/ σ2σ2 0
σ2 + σ2 0
Finally, we get the conditional posterior PDF,
fM|x(µ) = 1
σ
√
2π
2σ2 0
σ2 + σ2
exp −
0
1
µ −
2
σ2 µ0 + σ2 0x σ2 + σ2 2
/
0
σ2σ2 0
σ2 + σ2
,
0
and so we see that the posterior is a normal distribution with a mean that is a
weighted average of the prior mean and the observation x, and a variance that
is likewise a weighted average of the prior variance and the known variance of
the observable X.
This example, although quite simple, indicates that there can be many tedious
manipulations. It also illustrates why it is easier to work with PDFs without
normalizing constants.
We will now consider a more interesting example, in which neither µ nor
σ 2 is known. We will also assume that we have multiple observations.
Example 4.5 Normal with Inverted Chi-Squared and Conditional
Normal Priors
Suppose we assume the observable random variable X has a N(µ, σ 2 ), and we
wish to make inferences on µ and σ 2 . Let us assume that µ is a realization
of an unobservable random variable M ∈ IR and σ 2 is a realization of an
unobservable random variable Σ 2 ∈ IR+.
Theory of Statistics c○2000–2013 James E. Gentle