beamer - Vrije Universiteit Amsterdam

Mean Return Times and Probabilities
.
Definition
.
For all states j ∈ I:
.
Note that
πj = 1
.
▶ For transient and nulrecurrent states j: πj = 0;
▶ For positive recurrent states j:
µjj = 1 + ∑
i̸=j
µjj
pji[inf{n : Xn = j}|X0 = i] ≥ 1 ⇒ 0 < πj ≤ 1
▶ We will see that for irreducible positive recurrent sets R, (πj)j∈R forms a
probability mass function; i.e., ∑
j∈R
πj = 1.
c⃝ Ad Ridder (VU) SOR– Fall 2012 13 / 36