Statistical Methods in Medical Research 4ed

16.2 Prior and posterior distributions 537
improper prior is used, there is no guarantee that a proper posterior distribution
will result.
Another way to model ignorance is to use a uniform distribution over a finite
range. The finite range means that the value of p…u† can be chosen so that
the distribution integrates to one. The improper prior, mentioned above, can
be obtained as the limit as the range tends to infinity. The uniformity of the
distribution means that no one value is preferred over any other, which might
seem as good an implementation of ignorance as one might expect. However, the
use of a uniform distribution to model ignorance is not as satisfactory as it might
at first appear. Suppose u is a scalar. If there is ignorance about the value of u,
then surely there is equal ignorance about any function of u, such as u2 or
exp( u). However, a uniform distribution for u implies a non-uniform distribution
for these (and most other) functions of u.
A similar problem applies when the range of the parameter is infinite. The
discussion of improper priors given above tacitly assumed that the range of the
parameters is the whole real line. However, for many parameters, this is not true;
for example, a standard deviation, s, is necessarily positive. However, log s can
take any value and an improper prior for this quantity that is constant on the
whole real line transforms to a prior p…s† ˆ1=s for the standard deviation. This
form of improper prior is widely used for parameters that are necessarily
positive.
A way to define prior distributions that express ignorance and are invariant
under transformations of the parameters was proposed by Jeffreys (1961, p. 181).
For scalar u the Jeffreys prior is taken to be proportional to the square root of
the Fisher information, i.e.
…
q
p…u† /
2
" ( ) # 1=2
2 log p…xju† p…xju† dx : …16:3†
qu
When u is a vector, the Jeffreys prior is, strictly speaking, proportional to the
square root of the determinant of the matrix whose (i, j)th element is
…
q 2
( )
log p…xju† p…xju† dx:
quiquj
The expression for vector u is often cumbersome and it is quite common to form
the Jeffreys prior separately for each component of u using (16.3) and then use
the product of these as the prior for u. The argument here is that, broadly
speaking, ignorance is consistent with independence, so forming the joint prior
in this way is reasonable.
Jeffreys priors can lead to some slightly unusual results. If the data have a
univariate normal distribution, with mean m and standard deviation s, then the
following can be obtained: