Lecture Notes - Department of Mathematics and Statistics - Queen's ...

20 CHAPTER 3. CLASSIFICATION OF MARKOV CHAINS
and
Fπ ′ (i) = G[π ′ (j), j ∈ X \ {i}]
Let us define J := {i : π(i) ≥ π ′ (i)}, then for some linear functions with positive coefficients:
F J [π(j), j ∈ J] = G −J [π(j), j ∈ X \ J]
F J [π ′ (j), j ∈ J] = G −J [π ′ (j), j ∈ X \ J]
and by linearity, the same should hold for the difference:
F J [π(j) − π ′ (j), j ∈ J] = G −J [π(j) − π ′ (j), j ∈ X \ J]
But, the terms in the left hand vector are all non-negative, and the terms on the other hand are negative (since
the sums in π(i) equals 1, the terms on the right hand side should be strictly negative). Since the matrices
multiplying the vectors are all positive (this is where irreducibility is used), this is a contradiction. We skip the
proof of why the matrix elements are all positive -this follows from the equation π = πP and that all entries in
P are non-zero. As such, the entries in the measures should be identical to each other.
⊓⊔
Invariant Distribution via Dobrushin’s Ergodic Coefficient
One interesting result we have observed is the following. For every positive recurrent state i
lim P n (i, i) = 1
n→∞ E i [τ i ] .
Consider the iteration π t+1 = π t P. We would like to know when this iteration converges to a limit. We first
review some notions from analysis.
Review of Vector Spaces
Definition 3.2.1 A linear space is a space which is closed under addition and multiplication by a scalar.
Definition 3.2.2 A normed linear space X is a linear vector space on which a functional (a mapping from X to
R, that is a member of R X ) called norm is defined such that:
• ||x|| ≥ 0 ∀x ∈ X, ||x|| = 0 if and only if x is the null element (under addition and multiplication) of X.
• ||x + y|| ≤ ||x|| + ||y||
• ||αx|| = |α|||x||, ∀α ∈ R, ∀x ∈ X
Definition 3.2.3 In a normed linear space X, an infinite sequence of elements {x n } converges to an element x
if the sequence {||x n − x||} converges to zero.
Definition 3.2.4 A metric defined on a set X, is a function d : X × X → R such that:
• d(x, y) ≥ 0, ∀x, y ∈ X and d(x, y) = 0 if and only if x = y.
• d(x, y) = d(y, x), ∀x, y ∈ X.
• d(x, y) ≤ d(x, z) + d(z, y), ∀x, y, z ∈ X.
Definition 3.2.5 A metric space (X, d) is a set equipped with a metric d.
A normed linear space is also a metric space, with metric
d(x, y) = ||x − y||.