Theory of Statistics - George Mason University

4
Bayesian Inference
We have used an urn process to illustrate several aspects of probability and
sampling. An urn that contains balls of different colors can be used to illustrate
a primitive notion of probability – “What is the probability of drawing a red
ball?” – that can be integrated into our axiomatic development of probability
(as a set measure). Almost 250 years ago Pierre-Simon Laplace, the French
mathematician and astronomer, considered the urn problem and asked a very
different question: “Given that there are n balls in the urn, some of which are
red, if the first ball drawn is red, what is the probability that the proportion
of red balls, P, is p0 (some constant)?” While this form of question may be
quite natural to a layman, it is not consistent with our notion of probability.
There is a fixed number of red balls in the urn; the proportion P is either p0
or it is not.
Even if we adhere to our definitions of “probability”, we should be able
to rephrase this question into one for which statistical decision theory should
provide an answer. We might feel more comfortable, however, using different
words, and maybe even asking about “subjective probability” or “belief”
about the proportion of red balls in the urn. Laplace went on to answer the
question in a manner that we will identify later as a systematic approach to
such problems:
Pr(P = p0|first ball red) =
=
p0/(n − 2)
(n−1)/n
p=2/n
p/(n − 2)
p0
n(n − 1)/2 − 1 .
Another question that Laplace addressed concerned the probability π that
a human birth would be male. From the point of view that this is a random
process, the word “probability” in this context is consistent with our understanding
of the word. Laplace, however, went on to pose the question, “What
is the probability that the probability π is less than or equal to one half?”
Whether he felt it was relevant or not, he did not remark on the differences
Theory of Statistics c○2000–2013 James E. Gentle