IEOR 263B, Homework 2 Solution

IEOR 263B, Homework 2 Solution
Due February 5, 2009
1 Problems 2.22, 2.24
See solutions in the textbook.
2 Problem 4.30
Let Z i = X − Y i . We have P(Z i = 1) = P 1 (1 − P 2 ) := p and P(Z i = −1) = (1 − P 2 )P 1 := q. and P(Z i = 0) =
1 − p − q. Let S n = ∑ n
i=1 Z i. The problem to find the probability of error is similar to gambler’s ruin problem
even though we have p + q < 1. This is because the transition to same state does not effect the probability
whether M is reached before M − 1 or vice versa. It just effect the number of steps required. Hence:
P(error) = P(Up M before down –M) = 1 − (p/q)m
1 − (p/q) 2m = 1
1 + (p/q) m = 1
1 + λ m
where λ = p q = P 1(1 − P 2 )
P 2 (1 − P 1 ) .
Also by Wald’s identity (N is a stopping time)
3 Problem 4.31
E[N] = E[S N]
E[Z i ]
=
=
M(λ M − 1)
(P 1 − P 2 )(λ M + 1)
1
λM
(−M) +
1 + λM 1 + λ M M
p − q
The three states are: A (F in 1, S in 2), B (F in 2, S in 1) and C (hunting ends). Then the transition matrix
is given by
⎡
⎤
0.28 0.18 0.54
P = ⎣ 0.18 0.28 0.54 ⎦ . (1)
0 0 1
(a) The required probability is given by P (n) (1, 1) (i.e. the (1, 1) entry of the matrix P (n) ). To get the
explicit value, we first diagonalize the matrix P:
⎡
1 −1.5 −2.5
⎤ ⎡
1 0 0
⎤ ⎡
0.28 0.18 0.54
⎤
P = Γ −1 ΛΓ = ⎣ 1 1.5 −2.5 ⎦ · ⎣ 0 0.1 0 ⎦ · ⎣ 0.18 0.28 0.54 ⎦ . (2)
1 0 0 0 0 0.46 0 0 1
Hence P (n) = Γ −1 Λ (n) Γ, which is easy to compute. The (1, 1) entry of the computed matrix is (0.1 n +
0.46 n )/2.
(b) Notice that in both states A and B, the probability of ending is 0.54.
Geom(0.54), and the average time is thus 1/0.54 ≈ 1.85.
Hence the hunting time is ∼
1

4 Problem 4.38
We show that the Markov chain is time reversible with probabilities π i = a i /A where A = ∑ i a i. See that:
π i P ij =
a ia j q ij
A(a i + a j ) = π jP ji
5 Problem 4.39
Conditioning on the number of individuals that die in a given period it is easy to see:
i∑
( ) i
P ij =
p k (1 − p) i−k a
k
j−i+k
k=0
where a i = 0 if i < 0, and a i = e −λ λ i /i! if i ≥ 0.
The stationary probabilities as given in example 4.3(D) are:
For time reversibility to hold:
k=0
π i = e−λ/p (λ/p) i
, i = 0, 1, . . .
i!
π i P ij = π j P ji
⇐⇒ e−λ/p (λ/p) i i∑
( ) i
p k (1 − p) i−k a
i!
k
j−i+k = e−λ/p (λ/p) j
j!
⇐⇒ (λ/p)i−j j!
i!
i∑
( i
k
k=0
)
p k (1 − p) i−k a j−i+k =
j∑
( j
k
k=0
j∑
( j
k
k=0
)
p k (1 − p) i−k a i−j+k
)
p k (1 − p) i−k a i−j+k
Without loss of generality assume j > i. Then the R.H.S. of the above equation is:
j∑
( ) j
p k (1 − p) i−k a
k
i−j+k
L.H.S. is:
=
=
k=0
j−i−1
∑
k=0
j∑
k=j−i
=
=
( j
k
( j
k
λ/p) i−j j!
i!
i∑
k=0
i∑
(
k=0
)
p k (1 − p) i−k · 0 +
)
p k (1 − p) i−k a i−j+k
k=0
j∑
k=j−i
( j
k
)
p k (1 − p) i−k a i−j+k
i∑ i!
k!(i − k)! pk (1 − p) i−k e −λ λ j−i+k
(j − i + k)!
j!
k!(i − k)!(j − i + k)! pj−i+k (1 − p) i−k e −λ λ k
j
j − i + k
)
p j−i+k (1 − p) i−k −λ λk
e
k!
Changing the variable y = j − i + k we get
j∑
( )
j
p y (1 − p) j−y e −λ λ i−j+y
y
which is equal to R.H.S.
y=j−i
j∑
( j
=
y
y=j−i
(i − j + y)!
)
p y (1 − p) j−y e −λ a i−j+y
2

6 Problem 4.41
(a) By symmetry (or by noting that the chain is doubly stochastic), π j = 1 n
, j = 1, . . . , n. Hence the transition
probability of reverse chain is:
⎧
⎨ p, j = i − 1
Pij ∗ = π j P ji /π i = P ji = 1 − p, j = i + 1 ,
⎩
0, otherwise
where “0” refers to state n, and n + 1 refers to state 1.
(b) The Markov chain is not time reversible unless p = 0.5
7 Problem 4.44
All we need to show is that:
¯π i ¯Pij = ¯π j ¯Pji , for all i ≠ j
Substituting and simplifying this reduces to π j P ij = π j P ji which follows since the original untruncated Markov
chain is time reversible.
3