IEOR 263B, Homework 4 Solution

IEOR 263B, Homework 4 Solution
Due February 20, 2009
1 Problem 5.3
(i) Imagine that when in state i, dummy (or false) transitions which leave the system in state i occur at rate
M − v i . Then all transitions (real plus dummy) occur at a Poisson rte M and so the number of such
transitions by any time t is finite. Hence, the number of real transitions is also finite with probability 1.
(ii) let X n+1 denotes the time between the n th and n + 1 th transition and let J n denote the state at time n.
If we let
N(t) = sup{n : X 1 + · · · + X n ≤ t}
then N(t) denote the number of transitions by time t. Let j be the first recurrent state that is reached
and suppose it was reached at n 0 transition (which is finite by our assumption). Let n 1 , n 2 , . . . be the
successive integers n at which J n = j. Such integers exist as j is recurrent. Set T 0 = X 1 + · · · + X n0 and
T k = X nk−1 + · · · + X nk . T k is the time between k − 1 th and k th visit to j. It follows that {T k , k ≥ 1}
forms a renewal process and hence ∑ k≥1 T k = ∞ with probability 1. But we have:
∞∑ ∞∑
X n = = ∞
n=1
Hence if the number of transitions are infinite then the time for these transitions is also infinite with
probability one.
2 Problem 5.4
Let T i denote the time to go from i to i + 1, i ≥ 0. Then ∑ N−1
i=0 T i is the time taken to go from 0 to N. As T i
is exponential with rate λ i and the time T i are independent we have:
Also
and
3 Problem 5.5
E
[
exp
(
s
N−1
∑
i=0
T i
)]
k=1
λ i
= Π N−1
i=0 E [exp (sT i)] = Π N−1
i=1
λ i − s
∑
T i =
N−1
E
i=0
N−1
Var
For 0 ≤ s 1 ≤ s 2 . . . ≤ s n ≤ t, we want to calculate
∑
T i =
i=0
N−1
∑
i=0
N−1
∑
P(S i = s 1 , S i+1 = s 2 , . . . , S i+k−1 = s k |X(t) = i + k, X(0) = i)
i=0
1
λ i
1
λ 2 i
which is same as
P(T i = s 1 , T i+1 = s 2 − s 1 , . . . , S i+k−1 = s k − s k−1 , T i+k > t − s k |X(0) = i)
P(X(t) = i + k|X(0) = i)
1

Using the independence of T m ’s and substituting their densities we get:
iλe −iλs1 (i + 1)λe −(i+1)λ(s2−s1) . . . (i + k − 1)λe −(i+k−1)λ(s i+k−s i+k−1 ) e −(i+k)λ(t−s i+k−1)
( )
i + k − 1
e
i − 1
−λti (1 − e −λt ) k
Simplifying we get:
k!Π k λe −λ(t−sn)
m=1
1 − e −λt
4 Problem 5.12
(i) Since P 0 =
state 0),
1/λ
1/λ + 1/mu = µ , it follows from renewal theory (renewal every time we come back to
λ + µ
N(t)
lim = α 0µ
t→∞ t λ + µ + α 1λ
λ + µ
(ii) The expected total time spent in state 0 by time t is
From this:
E[T 0 (t)] =
∫ t
0
P 00 (s)ds = µ
λ + µ t + λ
(
(λ + µ) 2 1 − e (λ+µ)t)
E[N(t)] = α 0 E[T 0 (t)] + α 1 (t − E[T 0 (t)]).
2