Theory of Statistics - George Mason University

4.6 Bayesian Confidence Sets 369
as the usual Lebesgue measure; in the one-dimensional continuous case, we
seek the shortest interval. In the Bayesian approach, we so something similar,
except we use the posterior density as a measure.
The mechanics of determining credible sets begin with the standard
Bayesian steps that yield the conditional distribution of the parameter given
the observable random variable. If the density exists, we denote it as fΘ|x.
At this point, we seek regions of θ in which fΘ|x(θ|x) is large. In general, the
problem may be somewhat complicated, but in many situations of interest
it is relatively straightforward. Just as in the frequentist approach, the identification
of the region often depends on pivotal values, or pivotal functions.
(Recall that a function g(T, θ) is said to be a pivotal function if its distribution
does not depend on any unknown parameters.)
It is often straightforward to determine one with posterior probability
content of 1 − α.
4.6.2 Highest Posterior Density Credible sets
If the posterior density is fΘ|x(θ|x), we determine a number c such that the
set
Cα(x) = {θ : fΘ|x(θ) ≥ cα} (4.63)
is such that Pr(Θ ∈ Cα|X = x) = 1 − α. Such a region is called a level 1 − α
highest posterior density or HPD credible set.
We may impose other conditions. For example, in a one-dimensional continuous
parameter problem, we may require that one endpoint of the interval
be infinite (that is, we may seek a one-sided confidence interval).
An HPD region can be disjoint if the posterior is multimodal.
If the posterior is symmetric, all HPD regions will be symmetric about x.
For a simple example, consider a N(0, 1) prior distribution on Θ and a
N(θ, 1) distribution on the observable. The posterior given X = x is N(x, 1).
All HPD regions will be symmetric about x. In the case of a symmetric density,
the HPD is the same as the centered equal-tail credible set; that is, the
one with equal probabilities outside of the credible set. In that case, it is
straightforward to determine one with posterior probability content of 1 − α.
4.6.3 Decision-Theoretic Approach
We can also use a specified loss function to approach the problem of determining
a confidence set.
We choose a region so as to minimize the expected posterior loss.
For example, to form a two-sided interval in a one-dimensional continuous
parameter problem, a reasonable loss function may be
⎧
⎨k1(c1
− θ) if θ < c1,
L(θ, [c1, c2]) = 0 if c1 ≤ θ ≤ c2,
⎩
k2(θ − c2) if θ > c2.
Theory of Statistics c○2000–2013 James E. Gentle