Theory of Statistics - George Mason University

Notes and Further Reading 139
and formulates two postulates that essentially require invariance of the limit
under any selections within the collective. This notion came to be called “statistical
probability”. Two years later, Keynes (1921) developed a concept of
probability in terms of the relative support one statement leads to another
statement. This idea was called “inductive probability”. As Kolmogorov’s axiomatic
approach (see below) came to define probability theory and statistical
inference for most mathematicians and statisticians, the disconnect between
statistical probability and inductive probability continued to be of concern.
Leblanc (1962) attempted to reconcile the two concepts, and his little book is
recommended as a good, but somewhat overwrought, discussion of the issues.
Define probability as a special type of measure
We have developed the concept of probability by first defining a measurable
space, then defining a measure, and finally defining a special measure as a
probability measure.
Define probability by a set of axioms
Alternatively, the concept of probability over a given measurable space could
be stated as axioms. In this approach, there would be four axioms: nonnegativity,
additivity over disjoint sets, probability of 1 for the sample space, and
equality of the limit of probabilities of a monotonic sequence of sets to the
probability of the limit of the sets. The axiomatic development of probability
theory is due to Kolmogorov in the 1920s and 1930s. In Kolmogorov (1956),
he starts with a sample space and a collection of subsets and gives six axioms
that characterize a probability space. (Four axioms are the same or similar
to those above, and the other two characterize the collection of subsets as a
σ-field.)
Define probability from a coherent ordering
Given a sample space Ω and a collection of subsets A, we can define a total
ordering on A. (In some developments following this approach, A is required
to be a σ-field; in other approaches, it is not.) The ordering consists of the
relations “≺”, “”, “∼”, “≻”, and “”. The ordering is defined by five axioms
it must satisfy. (“Five” depends on how you count, of course; in the five
laid out below, which is the most common way the axioms are stated, some
express multiple conditions.) For any sets, A, Ai, B, Bi ∈ A whose unions and
intersections are in A (if A is a a σ-field this clause is unnecessary), the axioms
are:
1. Exactly one of the following relations holds: A ≻ B, A ∼ B, or A ≺ B.
2. Let A1, A2, B1, B2 be such that A1 ∩ A2 = ∅, B1 ∩ B2 = ∅, A1 B1, and
A2 B2. Then A1 ∪ A2 B1 ∪ B2. Furthermore, if either A1 ≺ B1 or
A2 ≺ B2, then A1 ∪ A2 ≺ B1 ∪ B2.
3. ∅ A and ∅ ≺ Ω.
4. If A1 ⊇ A2 ⊇ · · · and for each i, Ai B, then ∩iAi B.
Theory of Statistics c○2000–2013 James E. Gentle