beamer - Vrije Universiteit Amsterdam

Recurrent States
.
Theorem 3.5.3(a)
.
Suppose that R ⊂ I is an irreducible recurrent set of states; then fij = 1 for all
i, . j ∈ R.
Proof:
1. According to Lemma 3.5.2 it holds that i and j communicate; i.e,
> 0} ̸= ∅;
{m : p (m)
ji
2. Let r = min{m : p (m)
ji
> 0}. Then
∞∩
0 = 1 − fjj = ( {Xn ̸= j}|X0 = j)
≥ p (r)
ji (
n=1
∞∩
{Xn ̸= j}|X0 = i) = p (r)
(1 − fij).
n=1
c⃝ Ad Ridder (VU) SOR– Fall 2012 6 / 36
ji