Lecture Notes - Department of Mathematics and Statistics - Queen's ...

24 CHAPTER 3. CLASSIFICATION OF MARKOV CHAINS
Definition 3.3.3 A Markov chain is called recurrent if it is µ−irreducible and
whenever µ(A) > 0 and A ∈ B(X).
∞∑
∞∑
E x [ 1 xt∈A] = P t (x, A) = ∞, ∀x ∈ X,
t=1
t=1
While studying chains in an infinite state space, we use a stronger recurrence definition: A set A ∈ B(X) is
Harris recurrent if the Markov chain visits A infinitely often with probability 1, when the process starts in A:
Definition 3.3.4 A set A ∈ B(X) is Harris recurrent if
P x (η A = ∞) = 1, A ∈ B(X), ∀x ∈ A, (3.11)
Theorem 3.3.1 Harris recurrence of a set A is equivalent to
P x [τ A < ∞] = 1, ∀x ∈ A.
Proof: We now prove this argument ([Meyn-Tweedie, Chapter 9]): let τ A (1) be the first time the state hits
A. By the Strong Markov Property, the Markov chain sampled at successive intervals τ A (1), τ A (2) and so on is
also a Markov chain. Let Q be the transition kernel for this sampled Markov Chain. Now, the probability of
τ A (2) < ∞ can be computed recursively as
P(τ A (2) < ∞)
∫
= Q (x xτA (1) τ A(1), dy)P y (τ A (1) < ∞)
A
( ∫
)
= Q (x xτA (1) τ A(1), dy) P y (τ A (1) < ∞)
A
= 1 (3.12)
By induction, for every n ∈ Z +
P(τ A (n + 1) < ∞)
∫
= Q (x xτA (1) τ A(1), dy)P y (τ A (n) < ∞)
Now,
A
= 1 (3.13)
P x (η A ≥ k) = P x (τ A (k) < ∞),
since k times visiting a set requires k times returning to a set, when the initial state x is in the set. As such,
P x (η A ≥ k) = 1, ∀k ∈ Z +
is identically equal to 1. Define B k = {ω ∈ Ω : η(ω) ≥ k}, and it follows that B k+1 ⊂ B k . By the continuity of
probability P(∩B k ) = lim k→∞ P(B k ), it follows that P x (η A = ∞) = 1.
The proof of the other direction for showing equivalence is left as an exercise to the reader.
⊓⊔
Definition 3.3.5 A Markov Chain is Harris recurrent if the chain is µ−irreducible and every set A ⊂ X is
Harris recurrent whenever µ(A) > 0. If the chain admits an invariant probability measure, then the chain is
called positive Harris recurrent.
Remark: Harris recurrence is stronger than recurrence. In one, an expectation is considered, in the other, a
probability is considered. Consider the following example: Let P(1, 1) = 1 and P(x, x + 1) = 1 − 1/x 2 and
P(x, 1) = 1/x 2 . P x (τ 1 = ∞) = ∏ t≥x (1 − 1/t2 ) > 0. This chain is not Harris recurrent. It is π−irreducible for
the measure π(A) = 1 if 1 ∈ A. Hence, this chain is recurrent but not Harris recurrent.
⊓⊔