Lecture Notes - Department of Mathematics and Statistics - Queen's ...

28 CHAPTER 3. CLASSIFICATION OF MARKOV CHAINS
Likewise,
∫
〈µ T , Pf〉 :=
∫
µ T (dx)(
P(dy|x)f(y)) → 〈µ ∗ , Pf〉.
Now,
(µ T − µ T P)(f) = 1 ( T∑ −1
T E µ 0
Px k f −
k=0
T∑
−1
k=0
)
Px
k+1 f
= 1 )
T E µ 0
(f(x 0 ) − f(x T ) → 0. (3.15)
Thus,
(µ T − µ T P)(f) = 〈µ T , f〉 − 〈µ T P, f〉 = 〈µ T , f〉 − 〈µ T , Pf〉 → 〈µ ∗ − µ ∗ P, f〉 = 0.
Thus, µ ∗ is an invariant probability measure.
⊓⊔
Lasserre [19] gives the following example to emphasize the importance of the Feller property: Consider a Markov
chain evolving in [0, 1] given by: P(x, x/2) = 1 for all x ≠ 0 and P(0, 1) = 1. This chain does not admit an
invariant measure.
Remark 3.4.1 Lasserre presents a class of chains, Quasi-Feller chains, which are weak Feller except on a closed
set of points N such that P(x, N) = 0 for all x ∈ D = X \ N, P(x, N) < 1 for x ∈ N, and for all x ∈ D, the
chain is weak Feller.
3.4.2 Case without the Feller condition
One can also relax the weak Feller condition.
Theorem 3.4.2 [20] For a Markov chain, under a uniform countable additivity condition as in (4.7) in the next
chapter, there exists an invariant probability measure if and only if there exists a non-negative function g with
lim
n→∞
inf g(x) = ∞
x/∈K n
and an initial probability measure ν 0 such that sup t E ν0 [g(x t )] < ∞.
The proof of this result follows from a similar observation as (3.15) but with weak convergence replaced by
setwise convergence:
Lemma 3.4.3 [7, Thm. 4.7.25] Let µ be a finite measure on a measurable space (T, A). Assume a set of
probability measures Ψ ⊂ P(T) satisfies
Then Ψ is setwise precompact.
P ≤ µ, for all P ∈ Ψ.
A sequence of occupation measures under (4.7) has a subsequence which converges setwise and the limit of this
subsequence is invariant. As an example, consider a system of the form:
x t+1 = f(x t ) + w t (3.16)
where w t admits a distribution with a bounded density function, which is positive everywhere. If the system has
a finite second moment, then, the system admits an invariant probability measure which is unique.