CHAPTER 2: Markov Chains (part 3)

we have
v 2 = 13.3333, v 3 = 6.3333, v 4 = 16.3333
is
Example[understanding T] Consider a Markov Chain whose transition probability matrix
P =
1 2 3
1 a b c
2 d e f
3 0 0 1
The MC starts at time 0 with X 0 = 0. Let T = min{n ≥ 0 : X n = 3} Find P (X 3 = 0|X 0 =
0; T > 3)
Example [understanding one-step analysis] Consider a Markov Chain whose transition probability
matrix is
P =
The MC starts at time 0 with X0 = 2.
1 2 3 4
1 1 0 0 0
2 0.2 0.2 0.2 0.4
3 0.2 0.3 0.4 0.1
4 0 0 0 1
1. What is the probability that when the process is absorbed, it does so from state 2?
2. What is the probability that when the process is absorbed by state 4, it does so from state
2?
8