Hui's reference material on Markov chains (pdf)

More documents

Recommendations

Info

epresentation E(N i (k)|X 0 = i) = ∑ k l=0 P l (i, i), we obtain 0 = lim k→∞ ∑ k l=0 P l (i, i) k ≥ lim k→∞ ∑ k−m−n l=0 P l (j, j) · P n (i, j)P m (j, i) k k − m − n = lim · k→∞ k ∑ k−m−n l=0 P l (j, j) · P n (i, j)P m (j, i) k − m − n ∑ k l=0 = lim P l (j, j) · P n (i, j)P m (j, i) k→∞ k = P n (i, j)P m (j, i) m j and thus m j = ∞, which signifies the null recurrence of j. □ Thus we can call a communication class positive recurrent or null recurrent. In the former case, a construction of a stationary distribution is given in Theorem 13 Let i ∈ E be positive recurrent and define the mean first visit time m i := E(τ i |X 0 = i). Then a stationary distribution π is given by π j := m −1 i · ∞∑ P(X n = j, τ i > n|X 0 = i) n=0 for all j ∈ E. In particular, π i = m −1 i communication class belonging to i. Proof: First of all, π is a probability measure since ∑ j∈E n=0 ∞∑ P(X n = j, τ i > n|X 0 = i) = and π k = 0 for all states k outside of the = ∞∑ ∑ P(X n = j, τ i > n|X 0 = i) n=0 j∈E ∞∑ P(τ i > n|X 0 = i) = m i n=0 14
The particular statements in the theorem are obvious from theorem 8 and the definition of π. The stationarity of π is shown as follows. First we obtain π j = m −1 i · = m −1 i · = m −1 i · ∞∑ P(X n = j, τ i > n|X 0 = i) n=0 ∞∑ P(X n = j, τ i ≥ n|X 0 = i) n=1 ∞∑ P(X n = j, τ i > n − 1|X 0 = i) n=1 since X 0 = X τi = i in the conditioning set {X 0 = i}. Because of P(X n = j, τ i > n − 1|X 0 = i) = P(X n = j, τ i > n − 1, X 0 = i) P(X 0 = i) = ∑ P(X n = j, X n−1 = k, τ i > n − 1, X 0 = i) P(X 0 = i) k∈E = ∑ P(X n = j, X n−1 = k, τ i > n − 1, X 0 = i) P(X n−1 = k, τ i > n − 1, X 0 = i) k∈E × P(X n−1 = k, τ i > n − 1, X 0 = i) P(X 0 = i) = ∑ k∈E p kj P(X n−1 = k, τ i > n − 1|X 0 = i) we can transform further ∞∑ ∑ π j = m −1 i · p kj P(X n−1 = k, τ i > n − 1|X 0 = i) n=1 k∈E = ∑ k∈E p kj · m −1 i which completes the proof. □ ∞∑ n=0 P(X n = k, τ i > n|X 0 = i) = ∑ k∈E 15 π k p kj
Page 1 and 2: Chapter 2: Markov Chains and Queues
Page 3 and 4: Theorem 2 Let X denote a homogeneou
Page 5 and 6: n} depends only on the evolution of
Page 7 and 8: Proof: By definition, p ′ ij ∈
Page 9 and 10: Proof: Define τ (1) j := τ j and
Page 11 and 12: 3 Stationary Distributions Let X de
Page 13: which is a contradiction. □ For t
Page 17 and 18: Theorem 15 Let X denote an irreduci
Page 19 and 20: Proof: Due to theorem 16 it suffice
Page 21 and 22: ecurrent. □ An often difficult pr
Page 23 and 24: are black and the others are white.
Page 25 and 26: Exercise 14 Let X denote a Markov c
Page 27 and 28: for all k ∈ N 0 , A 0 , . . . , A
Page 29 and 30: Conditional probabilities are speci

Hui's reference material on Markov chains (pdf)

Create successful ePaper yourself

Delete template?

Save as template?