Theory of Statistics - George Mason University

332 4 Bayesian Inference
3. Identify the joint distribution; if densities exist, it is
fX,Θ(x, θ) = fX|θ(x)fΘ(θ). (4.16)
4. Determine the marginal distribution of the observable; if densities exist,
it is
fX(x) = fX,Θ(x, θ)dθ. (4.17)
Θ
This marginal distribution is also called the prior predictive distribution.
In slightly different notation, we can also write it as
fX(x) = fX|θ(x)fΘ(θ)dθ.
Θ
5. Determine the posterior conditional distribution of the parameter given
the observable random variable. If densities exist, it is
fΘ|x(θ) = fX,Θ(x, θ)/fX(x). (4.18)
This is the PDF of the distribution QH in equation (4.2), which is often
just called the “posterior”. The posterior conditional distribution is then
the basis for whatever decisions are to be made.
6. Assess the posterior conditional distribution in the light of prior beliefs.
This is called a sensitivity analysis. Repeat the steps above as appropriate.
These first steps in a Bayesian analysis involve identifying the components
in the equation
fX|θ(x)fΘ(θ) = fX,Θ(x, θ) = fΘ|x(θ)fX(x). (4.19)
Although we have written the PDFs above in terms of single random variables
(any of which of course could be vectors), in applications we assume we
have multiple observations on X. In place of fX|θ(x|θ) in (4.14), for example,
we would have the joint density of the iid random variables X1, . . ., Xn, or
fX|θ(xi|θ). The other PDFs would be similarly modified.
Given a posterior based on the random sample X1, . . ., Xn, we can form
a posterior predictive distribution for Xn+1, . . ., Xn+k:
fXn+1,...,Xn+k|x1,...,xn (xn+1, . . ., xn+k) =
Θ fX,Θ(xn+i, θ)dFΘ|x1,...,xn (θ).
(4.20)
Rather than determining the densities in equations (4.14) through (4.18)
it is generally sufficient to determine kernel functions. That is, we write the
densities as
fD(z) ∝ g(z). (4.21)
Theory of Statistics c○2000–2013 James E. Gentle