Exchangeable Sequences, Laws of Large Numbers, and the ...

Exchangeable Sequences,
Laws of Large Numbers,
and the Mortgage Crisis.
Myung Joo Song
Advisor: Prof. Jan Mandel
May 1 2009

Introduction
The law of large numbers for i.i.d. sequence gives convergence
of sample means to a constant, i.e., a deterministic quantity.
The yield from a mortgage can be understood as a random variable.
If a bank can make a large number of mortgages such their yields
are i.i.d., the average yield will converge to a deterministic quantity.
But events that appear to be independent may in fact be only exchangeable.
In a real financial market, there is always a fundamental factor which
can affect all the events at the same time, such as a war, or an economic
crisis which ruins the assumption of independence.

Independence of random variables
Definition:
( Ω Y P)
Let , , be a probability space. Then, the function
X : Ω → ℜ is a (real - valued) random variable if
( )
{ ω ω }
: X ≤ r ∈Y ∀r ∈ℜ.
Two random variables X and Y are independent iff
( ) ( ) ( )
( XY ) = X ⋅ Y
P X ≤ a, Y ≤ b = P X ≤ a ⋅P Y ≤ b ∀a,
b
{ } { } { }
then also, E E E
( X Y )
note: independent ⇒ uncorrelated (cov , =
0)
but the converse is not true.

Laws of Large Numbers
Weak Law of Large Numbers:
( 1 2 L)
( ) µ µ
Given X , X , an infinite sequence of i. i. d.
r.v.s with
E X = < ∞ , ∀ i ∈ Ν ,
i
1
P
X n = ( X 1 + L
+ X n ) → µ as n → ∞
n
( )
n
That is, lim P X − µ < ε = 1 for any ε .
n→∞

Laws of Large Numbers
Strong Law of Large Numbers:
( 1 2 L)
( ) µ µ
Given X , X , an infinite sequence of i. i. d.
r.v.s with
E X = < ∞ , ∀ i ∈ Ν ,
i
1
a. s.
X n = ( X 1 + L
+ X n ) → µ as n → ∞
n
( ) n µ
That is, P lim X = = 1.
n→∞

Exchangeability
Definition:
An infinite sequence of X , L , X , L of random
variables is said to be exchangeable if ∀ n ≥ 2,
D
( ) ( )
1 L n (1) L ( n )
X , , X = X , , X ∀ ∈ S ( n),
1
π π π
n
{ L
}
where S ( n) is the group of permutations of 1, , n .

Exchangeability
Example: Polya’s urn
An urn has initially r red and b black balls. Draw a ball at random and note
its color, and replace the ball back and add another ball of the same color.
th
Let X =1 if the i draw yields a red ball and X =0 otherwise.
i
Then,
r r + 1 b r + 2
P(
1,1,0,1 ) = ⋅ ⋅ ⋅
b + r b + r + 1 b + r + 2 b + r + 3
r b r + 1 r + 2
= ⋅ ⋅ ⋅ = P
b + r b + r + 1 b + r + 2 b + r + 3
Similarly for other cases.
The sequence X , L, X , L
is exchangeable.
1
n
i
( )
1,0,1,1 .

Conditional Expectation
( )
{ ω : ω }
( Ω )
Let X be r.v. on probability space , S, P . Given another
σ -algebra Y ⊂ S, a r.v. Y is called conditional expectation
of X given Y Y if Y is Y
Y -measurable, that is,
∈Ω Y ≤ a ∈Y ∀a ∈ℜ
and YdP = XdP ∀A∈ .
∫ ∫ Y
A A
( Y )
( Y )
In this case, denote Y = E X | .
Roughly speaking, E X | is averaging of X to the granularity
of Y Y (if YY
Y is finite, averaging on the atoms
of Y
).

LLN for exchangeable sequences
( ) n≥0
( P)
( )
n σ Ω n ⊂ n+
1
Let Y be a sequence of -algebras on , Y Y , . Y Y Y
Y
∀n ≥ 0. Then, a sequence of r.v.s X is called a martingale if,
{ n }
( i) E X < ∞,
each n;
( ii) X is Y -measurable, each n;
n n
{ Y }
n n≥0
( iii )E X | Y = X a . s ., each m ≤ n .
n m m
Martingale Convergence Theorem:
( X n ) { X n }
Let be a martingale s.t. sup E < ∞.
n≥1
n
1
Then lim X n =
X exists a. s. ( and is finite a. s.). Moreover, X is in L .
n→∞

LLN for exchangeable sequences
Recall
{ }
X i.i.d., Ε ⎡ X ⎤ < +∞
i
{ }
⎣ 1 ⎦
N 1
⇒ ∑ X i → Ε X
N
Theorem:
i=
1
[ ]
X exchangeable, Ε ⎡ X ⎤ < +∞
i
1
⇒ → Ε
N
1
⎣ 1 ⎦
a.s. , a deterministic number.
N
∑ X i
i=
1
[ X1
| Y Y ] a.s. , a random variable for some YY
Y Y

LLN for exchangeable sequences
proof : Let an infinite sequence X = ( X , X , L)
of random variables
n n+
1
1 2
be exchangeable and Let Y be the σ -algebra generated by all the n-
symmetric functions of X.
n
Y Y ⊇ YY
Y
If f is a measurable function for
which E ⎡⎣ X1 ⎤ ⎦ < +∞ , and if Y = g X
is bounded n-symmetric r.v., then for 1 ≤ j ≤ n ,
{ f ( X ) ( ) } ( ) ( )
j g X = f X1 g X j X 2 L X j− 1 X1 X j+
1 L
{ }
E E , , , , , ,
( ) ( )
{ f X g X }
E ,
⎧ n 1 ⎫
so that E ⎨ ∑ f ( X ) E { ( ) }
j Y ⎬ = f X1 Y .
n
( continued
)
=
⎩ j=
1 ⎭
1
( )

LLN for exchangeable sequences
proof : ( continued )
Then, take Y as the indicator of A,
1 , so that
1
n
∫ ∑ f ( X j ) dP = ∫ f ( X1 ) dP ( A∈
Y n )
n A j=
1
A
Then, by the definition of conditional expectation,
1
n j=
1
( j ) = E { ( 1)
| Y n}
f X f X
Since partial sums form a martingale, by the Martingale convergence
theorem,
n
∑
n
∞
1
lim ∑ f ( X ) = E { f ( X1 ) | Y Y ∞} a. s.
where Y Y ∞ = I Y
Y .
n→∞
n
A
j n
j= 1 n=
1
.
.

De Finetti’s Theorem
Definition :
Let X be r.v.s and let Y be a σ -field.
{ }
i
Say X is conditionally i . i . d . given Y if for A
{ }
i i
P X ∈ A , 1 ≤ i ≤ n | Y = ∏ P X ∈ A | Y
, and
( Y ) ( Y )
i i i i
i
( ) ( )
i Y Y j Y
Y
⊂ ℜ
P X ∈ A | = P X ∈ A | a.s., for each A, i ≠ j.

De Finetti’s Theorem
De Finetti's Theorem :
{ X } { X }
If is an exchangeable sequence then is conditionally i.i.d.
given
Proof
i i
Y
∞
: By the LLN of exchangeable sequence,
n
∞
1
lim ∑ f ( X ) = E { f ( X1 ) | Y Y ∞} a. s.
where Y Y ∞ = I Y
Y .
n→∞
n
j n
j= 1 n=
1
Let f y
⎧1
⎩0
y ≤ x
y > x
1
n j=
1
f X
1
n
j n X x
n 1
and lim ∑ f ( X j ) = E { f ( X1 ) | Y Y ∞} = P ( X1 ≤ x | Y
Y ∞ ) = F ( x)
,
n→∞
n
n
( ) = ⎨ . Then, ∑ ( j ) = # { ≤ ; j ≤ }
j=
1
( ) { ≤ Y
}
where F x =P X x | is a random distribution function. ( cont.)
1
∞

De Finetti’s Theorem
Proof : ( cont.) X , L X are i.i.d. ⇔ ∃ F distribution function s.t.
1
( ) ( ) ( )
( X ≤ x ) = I ( X )
P X ≤ x ∧L ∧ X ≤ x = F x ⋅L⋅ F x ∀x
, L,
x .
1 1 n n 1 n 1 n
{ }
−∞ x
Since P E ,
1 1 ( , ] 1
n
1
( ) ( 1 )
= F ( x x1 ) ⋅ L ⋅ F ( x xk
)
( ) ( ) Y
{ }
k ∞
P X ≤ x ∧L ∧ X ≤ x | ℑ = E I X ⋅L ⋅ I ( X ) | Y
1 1 k k ∞ ( −∞, x ] 1 ( −∞,
x ]
∀x , L, x : ω a F ω, x ⋅L⋅ F ω, x is ∞ -measurable.
1 k 1
k
{ { ( −∞, x ( 1]
1 ) ⋅L⋅ ( −∞, x ( k ] k ) Y } ∞ } = ( 1)
⋅L⋅ ( k )
F ⊂ Y ∞ ( xi ) F urable ∀ i = 1,2, Lk.
{ I( −∞, x ( ) ( )
1]
X1 ⋅L⋅ I( −∞,
xk ] X k F} = ( x1 ) ⋅L⋅ ( xk
)
( X ≤ x ∧L ∧ X ≤ x F ) = ( x ) ⋅L⋅ ( x )
k
{ }
E E I X I X | | F E F x F x | F
where , and F is -meas
Then, E | F F ,
and thus, P | F F .
1 1 k k 1
k

Application to the mortgage mess
Let a r.v. X be the payoff from mortgage i and bank wants to spread
i
the risk by making a large number of such mortgages and create a mortgage
pool with deterministic payoff.
However, if X are not i.i.d
but only exchangeable, there is some nontrivial
i
σσ -algebra Y Y Y Y that underlies them all, i.e. X are conditionally i.i.d. on Y
Y
Y .
Then the payoff seems to be (asymtotically) deterministic but is actually
random variable, Y -measurable, so its value changes depending on which
set in Y , the event ω is in.
Thus, there are only exchangeable sequences in reality, there is no such a
thing as i.i.d. sequence
since there are always some underlying assumptions
for which set S ∈Y we have ω ∈ S that can change .
i
a

References
Aldous, D. (1985). Exchangeability and related topics. In: École d'Été
de Probabilités de Saint-Flour XII— Hennequin P. L., ed. (1985)
Berlin: Springer. 1–198. Lecture Notes in Mathematics 1117
Doob, J.L. The development of rigor in mathematical probability (1900-
1950). Amer. Math. Monthly, 103(7):586-595, 1996. Reprinted from
Development of mathematics 1900-1950, edited by J.P.Pier, pp.157-
170, Birkhauser, Basel, 1994.
Kingman, J.F.C. Uses of exchangeability. Ann. Probability, 6(2):183-
197, 1978
Jacod, J. and Protter, P. Probability essentials. Universitext. Springer-
Verlag, Berlin, second edition, 2003
Shiryaev, A.N. Probability, Vol 95 of Graduate Texts in Mathematics.
Springer-Verlag, New York, second edition, 1996.