Linear Algebra, Theory And Applications, 2012a

282 MARKOV CHAINS AND MIGRATION PROCESSES
Proof: Let A be the above matrix and suppose Ax =λx. Then letting A ′ denote
⎛
⎞
0 p 0 ··· 0
q 0 p ··· 0
. 0 q 0 ..
. ,
⎜
.
⎝ . 0 .. . .. p ⎟
⎠
0
. 0 q 0
it follows
and so from the above theorem,
A ′ x =(λ − a) x
|λ − a| < 1.
Example 11.3.6 In the gambler’s ruin problem a gambler plays a game with someone, say
a casino, until he either wins all the other’s money or loses all of his own. A simple version
of this is as follows. Let X k denote the amount of money the gambler has. Each time the
game is played he wins with probability p ∈ (0, 1) or loses with probability (1 − p) ≡ q. In
case he wins, his money increases to X k +1 and if he loses, his money decreases to X k − 1.
The transition probability matrix P, describing this situation is as follows.
⎛
⎞
1 0 0 0 ··· 0 0
q 0 p 0 ··· 0 0
0 q 0 p ··· 0 .
P =
. 0 0 q 0 ..
. 0
.
.
. 0 .. . ..
p 0
⎜
⎝
⎟
0 0 . 0 q 0 p ⎠
0 0 0 0 0 0 1
(11.2)
Here the matrix is b +1× b + 1 because the possible values of X k are all integers from 0 up
to b. The 1 in the upper left corner corresponds to the gambler’s ruin. It involves X k =0
so he has no money left. Once this state has been reached, it is not possible to ever leave
it. This is indicated by the row of zeros to the right of this entry the k th of which gives the
probability of going from state 1 corresponding to no money to state k 1 .
In this case 1 is a repeated root of the characteristic equation of multiplicity 2 and all
the other eigenvalues have absolute value less than 1. To see that this is the case, note that
the characteristic polynomial is of the form
⎛
(1 − λ) 2 det
⎜
⎝
⎞
−λ p 0 ··· 0
q −λ p ··· 0
0 q −λ . .
. . .
.
. 0 .. . .. p ⎟
⎠
.
0 . 0 q −λ
1 No one will give the gambler money. This is why the only reasonable number for entries in this row to
the right of 1 is 0.