Linear Algebra, Theory And Applications, 2012a

284 MARKOV CHAINS AND MIGRATION PROCESSES
and so
α + β =1
( b
α + β q
p)
=0
{
1
Solution is : β = − ,α=
( }
p) q b
. Substituting this in to (11.4) and simplifying,
−1+( p) q b −1+( p) q b
yields the following in the case that p ≠ q.
Note that
P j = pb−j q j − q b
p b − q b (11.5)
p b−j q j − q b
lim
p→q p b − q b
= b − j .
b
Thus as the game becomes more fair in the sense the probabilities of winning become closer
to 1/2, the probability of ruin given an initial amount j is b−j
b .
Alternatively, you could consider the difference equation directly in the case where p =
q =1/2. In this case, you can see that two solutions to the difference equation
P j = 1 2 P j−1 + 1 2 P j+1 for j ∈ [1,b− 1] , (11.6)
P 0 = 1, and P b =0.
are P j = 1 and P j = j. This leads to a solution to the above of
P j = b − j . (11.7)
b
This last case is pretty interesting because it shows, for example that if the gambler
starts with a fortune of 1 so that he starts at state j =1, then his probability of losing all
is b−1
b
which might be quite large, especially if the other player has a lot of money to begin
with. As the gambler starts with more and more money, his probability of losing everything
does decrease.
11.4 Exercises
1. Suppose the migration matrix for three locations is
⎛
.5 0 .3
⎞
⎝ .3 .8 0 ⎠ .
.2 .2 .7
Find a comparison for the populations in the three locations after a long time.
2. Show that if ∑ i a ij =1, then if A =(a ij ) , then the sum of the entries of Av equals
the sum of the entries of v. Thus it does not matter whether a ij ≥ 0 for this to be so.
3. If A satisfies the conditions of the above problem, can it be concluded that lim n→∞ A n
exists?
4. Give an example of a non regular Markov matrix which has an eigenvalue equal to
−1.