Theory of Statistics - George Mason University

140 1 Probability Theory
5. Let U ∼ U(0, 1) and associate the ordering (≺, , ∼, ≻, ) with Lebesgue
measure on [0, 1]. Then for any interval I ⊆ [0, 1], either A ≻ I, A ∼ I, or
A ≺ I.
These axioms define a linear, or total, ordering on A. (Exercise 1.89).
Given these axioms for a “coherent” ordering on Ω, we can define a probability
measure P on A by P(A) ≤ P(B) iff A B, and so on. It can be shown
that such a measure exists, satisfies the Kolmogorov axioms, and is unique.
At first it was thought that the first 4 axioms were sufficient to define
a probability measure that satisfied Kolmogorov’s axioms, but Kraft et al.
(1959) exhibited an example that showed that more was required.
A good exposition of this approach based on coherency that includes a
proof of the existence and uniqueness of the probability measure is given by
DeGroot (1970).
Define probability from expectations of random variables
Although the measurable spaces of Sections 1.1.1 and 0.1 (beginning on
page 686) do not necessarily consist of real numbers, we defined real-valued
functions (random variables) that are the basis of further development of
probability theory. From the axioms characterizing probability (or equivalently
from the definition of the concept of a probability measure), we developed
expectation and various unifying objects such as distributions of random
variables.
An alternate approach to developing a probability theory can begin with
a sample space and random variables defined on it. (Recall our definition
of random variables did not require a definition of probability.) From this
beginning, we can base a development of probability theory on expectation,
rather than on a probability measure as we have done in this chapter. (This
would be somewhat similar to our development of conditional probability from
conditional expectation in Section 1.5.)
In this approach we could define expectation in the usual way as an integral,
or we can go even further and define it in terms of characterizing
properties. We characterize an expectation operator E on a random variable
X (and X1 and X2) by four axioms:
1. If X ≥ 0, then E(X) ≥ 0.
2. If c is a constant in IR, then E(cX1 + X2) = cE(X1) + E(X2).
3. E(1) = 1.
4. If a sequence of random variables {Xn} increases monotonically to a limit
{X}, then E(X) = limn→∞ E(Xn).
(In these axioms, we have assumed a scalar-valued random variable, although
with some modifications, we could have developed the axioms in terms of
random variables in IR d .) From these axioms, after defining the probability of
a set as
Pr(A) = E(IA(ω)),
Theory of Statistics c○2000–2013 James E. Gentle