SOLUTIONS MANUAL for Stochastic Modeling: Analysis and ...

82 CHAPTER 7. CONTINUOUS-TIME PROCESSES
the failure rate is 0.02/hour, as stated in the problem. When {Y t =1or2} the repair
rate is 1/24/hour ≈ 0.04/hour.
Therefore,
and
⎛
⎜
G = ⎝
−0.02 0.02 0
0.04 −0.06 0.02
0 0.04 −0.04
⎞
⎟
⎠
π 0 ≈ 0.57, π 1 ≈ 0.29, π 2 ≈ 0.14
(a) π 2 ≈ 0.14 or 14% of the time.
(b) π 1 + π 2 ≈ 0.43 or 43% of the time.
11. Let X ij be the total time in j given {Y 0 = i}.
Let Z j bethetimespentinj on a single visit.
Then
µ ij =E[X ij ]=E[Z j ]I(i = j)+ ∑
k∈T ,k≠i
E[X kj ]p ik
becauseifwestartinj we must spend Z j there, and we can condition on the first state
visited after state i; the Markov property implies that once the process leaves i for k,
the fact that it started in i no longer matters. But notice that
E[X kj ]=µ kj
Therefore,
µ ij =1/g jj I(i = j)+ ∑
k∈T ,k≠i
µ kj p ik
Then noting that p ik = g ik /g ii and g jj = g ii if i = j, wehave
µ ij =
I(i = j)
g ii
+ ∑
k∈T ,k≠i
µ kj
g ik
g ii
or
g ii µ ij = I(i = j)+
∑
k∈T ,k≠i
g ik µ kj
12. Let Z i be the time initially spent in state i.
Then
ν ij =E[V ij ]=E[Z i ]+ ∑
E[V kj ]p ik
k≠i,j