beamer - Vrije Universiteit Amsterdam

Finite Unichain Case
.
Ergodic Theorem Finite Case
.
Suppose that the Markov chain satisfies the unichain condition with a finite
set of transient states (|T| < ∞) and with a finite irreducible set of recurrent
states (|R| < ∞). Then
.
1
lim
n→∞ n
n∑
f (Xk) = ∑
πjf (j) (w.p. 1)
k=1
Proof: let r be an arbitrary initial state
1
n
n∑
k=1
f (Xk) = 1
n
k=1
j∈I
j∈I
n∑ ∑
{Xk = j|X0 = r}f (j) = ∑ (
1
n
j∈I
n∑
k=1
)
{Xk = j|X0 = r} f (j).
Take n → ∞; interchange limit and finite sum allowed; apply empirical state
average property (slide 12).
c⃝ Ad Ridder (VU) SOR– Fall 2012 29 / 36