Lecture Notes - Department of Mathematics and Statistics - Queen's ...

34CHAPTER 4. MARTINGALES AND FOSTER-LYAPUNOV CRITERIA FOR STABILIZATION OF MARKOV CHAINS
Proof: We observe that
τ∑
−1
E[X τ − X ρ |F ρ ] = E[ X k+1 − X k |F ρ ]
k=ρ
τ∑
−1
τ∑
−1
= E[ E[X k+1 − X k |F k ]|F ρ ] = E[ 0|F ρ ] = 0 (4.1)
k=ρ
k=ρ
⋄
Theorem 4.1.5 Let {X n } be a sequence of F n -adapted integrable real random variables. Then, the following
are equivalent: (i) (X n , F n ) is a sub-martingale. (ii) If T, S are bounded stopping times with T ≥ S (almost
surely), then E[X S ] ≤ E[X T ]
Proof: Let S ≤ T ≤ n Now, note that,
E[X T − X S |F S ] =
n∑
E[1 (T ≥k) 1 (S