Linear Algebra, Theory And Applications, 2012a

11.3. MARKOV CHAINS 283
and the factor after (1 − λ) 2 has zeros which are in absolute value less than 1. Its zeros are
the eigenvalues of the matrix
⎛
⎞
0 p 0 ··· 0
q 0 p ··· 0
.
A ≡
0 q 0 .. . ⎜
.
⎝ . 0 .. . .. p ⎟
⎠
0 . 0 q 0
and by Corollary 11.3.5 these all have absolute value less than 1.
Therefore, by Theorem 11.1.3 lim n→∞ P n exists. The case of lim n→∞ p n j0 is particularly
interesting because it gives the probability that, starting with an amount j, the gambler
eventually ends up at 0 and is ruined. From the matrix, it follows
p n j0 = qp n−1
(j−1)0 + ppn−1 (j+1)0
for j ∈ [1,b− 1] ,
p n 00 = 1, and p n b0 =0.
To simplify the notation, define P j ≡ lim n→∞ p n j0 as the probability of ruin given the initial
fortune of the gambler equals j. Then the above simplifies to
P j = qP j−1 + pP j+1 for j ∈ [1,b− 1] , (11.3)
P 0 = 1, and P b =0.
Now, knowing that P j exists, it is not too hard to find it from (11.3). This equation is
called a difference equation and there is a standard procedure for finding solutions of these.
YoutryasolutionoftheformP j = x j and then try to find x such that things work out.
Therefore, substitute this in to the first equation of (11.3) and obtain
x j = qx j−1 + px j+1 .
Therefore,
px 2 − x + q =0
and so in case p ≠ q, you can use the fact that p + q = 1 to obtain
x = 1 (
1+ √ )
(1 − 4pq)
2p
= 1
2p
= 1 or q p .
or 1
2p
)
(
1+ √ (1 − 4p (1 − p))
(
1 − √ )
(1 − 4pq)
or 1 (
1 − √ )
(1 − 4p (1 − p))
2p
Now it follows that both P j = 1 and P j =
Therefore, anything of the form
(
q
p
) j
satisfy the difference equation (11.3).
( ) j q
α + β
(11.4)
p
will satisfy this equation. Find a, b such that this also satisfies the second equation of (11.3).
Thus it is required that
( ) b q
α + β =1,α+ β =0
p