Hui's reference material on Markov chains (pdf)

Corollary 2 For all i, j ∈ E the relations
r jj = (1 − f jj ) −1 and r ij = f ij r jj
hold, with the conventions 0 −1 := ∞ and 0 · ∞ := 0 included. In particular, the
expected number r jj of visits to the state j ∈ E is finite if j is transient and infinite
if j is recurrent.
Theorem 8 Recurrence and transience of states are class properties with respect
to the relation ↔. Furthermore, a recurrent communication class is always
closed.
Proof: Assume that i ∈ E is transient and i ↔ j. Then there are numbers
m, n ∈ N with 0 < P m (i, j) ≤ 1 and 0 < P n (j, i) ≤ 1. The inequalities
∞∑
∞∑
∞∑
P k (i, i) ≥ P m+h+n (i, i) ≥ P m (i, j)P n (j, i) P k (j, j)
k=0
h=0
now imply r jj < ∞ because of representation (7). According to corollary 2 this
means that j is transient, too.
If j is recurrent, then the same inequalities lead to
r ii ≥ P m (i, j)P n (j, i)r jj = ∞
which signifies that i is recurrent, too. Since the above arguments are symmetric
in i and j, the proof of the first statement is complete.
For the second statement assume that i ∈ E belongs to a communication class
C ⊂ E and p ij > 0 for some state j ∈ E \ C. Then
f ii = p ii + ∑ h≠i
p ih f hi ≤ 1 − p ij < 1
k=0
according to formula (6), since f ji = 0 (otherwise i ↔ j). Thus i is transient,
which proves the second statement.
□
Theorem 9 If the state j ∈ E is transient, then lim n→∞ P n (i, j) = 0, regardless
of the initial state i ∈ E.
Proof: If the state j is transient, then the first equation in corollary 2 yields r jj <
∞. The second equation in the same corollary now implies r ij < ∞, which by
the representation (7) completes the proof.
□
10