Theory of Statistics - George Mason University

114 1 Probability Theory
Exchangeability, Conditioning, and Independence
De Finetti’s representation theorem (Theorem 1.30 on page 75) requires an
infinite sequence, and does not hold for finite sequences. For example, consider
an urn containing one red ball and one blue ball from which we draw the balls
without replacement. Let Ri = 1 if a red ball is drawn on the i th draw and
Ri = 0 otherwise. (This is the Polya’s urn of Example 1.6 on page 24 with
r = b = 1 and c = −1.) Clearly, the sequence R1, R2 is exchangeable. Because
Pr(R1 = 1, R2 = 1) = 0,
if there were a measure µ as in de Finetti’s representation theorem, then we
would have
0 =
1
0
π 2 dµ(π),
which means that µ must put mass 1 at the point 0. But also
which would mean that
Pr(R1 = 0, R2 = 0) = 0,
0 =
1
0
(1 − π) 2 dµ(π).
That would not be possible if µ satisfies the previous requirement. There are,
however, finite versions of de Finetti’s theorem; see, for example, Diaconis
(1977) or Schervish (1995).
An alternate statement of de Finetti’s theorem identifies a random variable
with the distribution P, and in that way provides a more direct connection
to its use in statistical inference.
Theorem 1.61 (de Finetti’s representation theorem (alternate))
of binary random variables is exchangeable iff there
The sequence {Xi} ∞ i=1
is a random variable Π such that, conditional on Π = π, the {Xi} ∞ i=1 are
iid Bernoulli random variables with parameter π. Furthermore, if {Xi} ∞ i=1
is exchangeable, then the distribution of Π is unique and Xn = n
i=1 Xi/n
converges to Π almost surely.
Example 1.30 exchangeable Bernoulli random variables that are
conditionally iid Bernoullis (Schervish, 1995)
Suppose {Xn} ∞ n=1 are exchangeable Bernoulli random variables such that for
each n and for k = 0, 1, . . ., n,
n
Pr = k = 1
n + 1 .
i=1
Now Xn a.s.
→ Π, where Π is as in Theorem 1.61, and so Xn d → Π. To determine
the distribution of Π, we write the CDF of Xn as
Theory of Statistics c○2000–2013 James E. Gentle