SOLUTIONS MANUAL for Stochastic Modeling: Analysis and ...

y conditioning on the first state visited after i (if it is not j), and recalling that the
Markov property implies that once the process leaves i for k the fact that it started in
i no longer matters. But notice that
E[V kj ]=ν kj
83
Therefore,
ν ij =1/g ii + ∑
k≠i,j
ν kj p ik
Then noting that p ik = g ik /g ii
ν ij =1/g ii + ∑
or
ν kj g ik /g ii
k≠i,j
g ii ν ij =1+ ∑
g ik ν kj
k≠i,j
13. (a) We use the embedded Markov chain and holding times to parameterize the Markov
process. The state space and embedded Markov chain are given in the answer to
Exercise 27, Chapter 6.
We take t in minutes, so that 1/ψ 1 =1/2, 1/ψ 2 =1, 1/ψ 3 =2, 1/ψ 4 =1andψ 5
is ∞. Therefore,
⎛
G =
⎜
⎝
=
⎜
⎝
−ψ 1 0.5ψ 1 0.4ψ 1 0.1ψ 1 0
0 −ψ 2 0.05ψ 2 0.05ψ 2 0.9ψ 2
0 0.05ψ 3 −ψ 3 0.05ψ 3 0.9ψ 3
0 0.05ψ 3 0.05ψ 3 −ψ 4 0.9ψ 4
0 0 0 0 0
⎛
−2 1 0.8 0.2 0
0 −1 0.05 0.05 0.9
0 0.025 −0.5 0.025 0.45
0 0.05 0.05 −1 0.9
0 0 0 0 0
(b) We might expect the time to perform each type of transaction to be nearly constant,
implying that the exponential distribution is not appropriate. The only
way to be certain, however, is to collect data.
(c) We need 1 − p 15 (t) fort = 4 minutes. Using the uniformization approach with
n ∗ = 16 and carrying 10 digits of accuracy in all calculations,
with truncation error ≤ 0.004.
1 − ˜p 15 (4) = 1 − 0.88 = 0.12
⎞
⎟
⎠
⎞
⎟
⎠