Lecture Notes - Department of Mathematics and Statistics - Queen's ...

26 CHAPTER 3. CLASSIFICATION OF MARKOV CHAINS
Theorem 3.3.3 (Theorem 5.5.3 of [25], lemma 17 [28] ) For an aperiodic and irreducible Markov chain
{x t } every petite set is small.
A maximal irreducibility measure ψ is an irreducibility measure such that for all other irreducibility measures
φ, we have ψ(B) = 0 ⇒ φ(B) = 0 for any B ∈ B(X). Define B + (X) = {A ∈ B(X) : ψ(A) > 0} where ψ is a
maximal irreducibility measure.
Theorem 3.3.4 For an aperiodic and irreducible Markov chain every petite set is petite with a maximal irreducibility
measure for a distribution with finite mean.
Nummelin’s Splitting Technique
The results on recurrence apply to uncountable chains with no atom provided there is a small set or a petite set.
Suppose a set A is 1-small. Define a process z t = (x t , a t ), z t ∈ X × {0, 1}. That is we enlarge the probability
space. Suppose that when x t /∈ A, (a t , x t ) evolve independently from each other. However, when x t ∈ A, we
pick a Bernoulli random variable, and with probability δ the state visits A × {1} and with probability 1 −δ visits
A × {0}. From A × {1}, the transition for the next time stage is given by ν(dxt) and from A × {0}, it visits the
future time stage with probability
P(dx t+1 |x t ) − ν(dx t+1 )
.
1 − δ
Now, pick δ = ν(X). In this case, A×{1} is an accessible atom, and one can verify that the marginal distribution
of the original Markov process {x t } has not been altered.
The following can be established using the above construction.
δ
Proposition 3.3.1 If
then,
sup E[min(t > 0 : x t ∈ A)|x 0 = x] < ∞
x∈A
sup E[min(t > 0 : z t ∈ (A × {1}))|z 0 = z] < ∞.
z∈(A×{1})
3.3.3 Existence of an Invariant Distribution
We state the final Theorem on the invariant distributions for Markov chains:
Theorem 3.3.5 (Meyn-Tweedie) Consider a Markov process {x t } taking values in X. If there is a Harris
recurrent set A which is also a µ− petite set for some positive measure µ, and if the set satisfies
sup E[min(t > 0 : x t ∈ A)|x 0 = x] < ∞,
x∈A
then the Markov chain is positive Harris recurrent and it admits a unique invariant distribution.
In this case, the invariant measure satisfies the following, which is a generalization of Kac’s Lemma [14]:
Theorem 3.3.6 For a µ-irreducible Markov chain with a unique invariant probability measure; the following is
the invariant probability measure:
∫
π(A) =
C
τ∑
C−1
π(dx)E x [ 1 {xk ∈A}], ∀A ∈ B(X), µ(A) > 0, π(C) > 0
k=0