SOLUTIONS MANUAL for Stochastic Modeling: Analysis and ...

68 CHAPTER 6. DISCRETE-TIME PROCESSES
f (n)
15
n
1 0
2 0.9
3 0.09
4 0.009
5 0.0009
6 0.00009
7–20 nearly 0
(d) We need 100µ 12 because µ 12 is the expected number of times in the withdraw
state.
T = {1, 2, 3, 4} R = {5}
M = (I − P TT ) −1
⎛
⎞
1.0 0.5291005291 0.4338624339 0.1481481481
0 1.005291005 0.05291005291 0.05291005291
= ⎜
⎟
⎝ 0 0.05291005291 1.005291005 0.05291005291 ⎠
0 0.05291005291 0.05291005291 1.005291005
Therefore, 100µ 12 ≈ 100(0.5291) = $52.91.
28. Clearly Pr{H =1} =1− p ii = γ. Suppose the result is correct for all a ≤ n for some
n>1. Then
Pr{H = n +1} = Pr{S n+1 ≠ i, S n = i, S n−1 = i,...,S 1 = i | S 0 = i}
= Pr{S n+1 ≠ i, S n = i,...,S 2 = i | S 1 = i, S 0 = i} Pr{S 1 = i | S 0 = i}
= Pr{S n ≠ i, S n−1 = i,...,S 1 = i | S 0 = i}(1 − γ)
= (1− γ) n−1 γ(1 − γ) =(1− γ) n γ
from the induction hypothesis. Therefore, it is correct for any a.
29. Since f (1)
jj = p jj , R (1)
BB = P BB .
We can write
f (2)
jj = ∑ h∈A
Pr{S 2 = j, S 1 = h | S 0 = j}
= ∑ h∈A
Pr{S 2 = j | S 1 = h, S 0 = j} Pr{S 1 = h | S 0 = j}