IEOR 263B, Homework 8 Solution

(Adapted from Yifen Chen’s solution)
1 Problem 1
IEOR 263B, Homework 8 Solution
Due April 16, 2009
(a) Assuming that Yi is exponential with mean 8 with T = 10, compute αt and α
αt = 1
�
� �
∞
αt+1F (αt+1) + ydF (y) =
1 + r
1
�
� ∞ � �
αt+1
αt+1F (αt+1) + ydF (y) − ydF (y)
1 + r
0
0
Then we have
αt+1
α ′ = [α1, α2, α3, α4, α5, α6, α7, α8, α9, α10]
= [17.8835, 17.3236, 16.6750, 15.9140, 15.0065, 13.8988, 12.4998, 10.6318, 7.8431, 0.0000]
By setting the total period as 500, we can find that the threshold α converge to 22.8871.
(b) Repeat part (a) when the mean is 14.
When mean=14, we have
α ′ = [α1, α2, α3, α4, α5, α6, α7, α8, α9, α10]
= [31.2961, 30.3162, 29.1812, 27.8497, 26.2614, 24.3228, 21.8747, 18.6057, 13.7255, 0.0000]
By setting the total period as 500, we can find that the threshold α converge to 40.0525.
(c) A number of you had problems in this part.
1. The prior probability of the mean is 8 (λ = 1/8)is ρ0 By Bayes’ rule, the posterior probability
ρt+1 = P [mean = 8|Y1, Y2, . . . , Yt+1] = 1 − P [mean = 14|Y1, Y2, . . . , Yt+1]
=
f(yt+1|λ = 1/8)ρt
f(yt+1|λ = 1/8)ρt + f(yt+1|λ = 1/14)(1 − ρt)
2. Using the approximation method, run the simulation N times. The number N should be large
enough to ensure that your calculation of E(Profit, ρ0 = x), x = 0.1, . . . , 1.0 has converged. For
example, compute the expected values for N = 1000, N = 5000, N = 10000, and see the percentage
change in the values. You could set a criterion such as “choose N such that increasing N changes
the expected values by less than 1%”. I don’t think any of you did a convergence study.
3. For sensitivity, you want to look at something like
E(Profit, ρ0 = 1.0) − E(Profit, ρ0 = 0.1)
,
E(Profit, ρ0 = 0.5)
i.e. the relative range size. A statement like “the profit is insensitive to the prior because the total
range is 2” is incorrect; as a counter-example, if the E(Profit, ρ0 = 0.5) = 0.1, then this range is
actually huge. A number of you plotted E(Profit, ρ0 = x), x = 0.1, . . . , 1.0 against ρ0, which is
good, but you really want to rescale the y-axis by E(Profit, ρ0 = 0.5) to get the relative comparison.
1

2 Problem 2
In each period, the amount of money the blackmailer already have is fixed and not relevant for decision
making. Hence, in each period, the blackmailer only need consider the future return. In the last period, the
value function of the blackmailer is
�
VN (uN ) = max R, max
uN ∈[0,1] {uN(1
�
− uN)} = max
�
� �
R, max − uN −
uN ∈[0,1]
1
�2 2
+ 1
��
4
Hence, we can see that in period N, the blackmailer will offer a lump sum R to the victim if R > 1/4, otherwise,
the blackmailer will demand a payment of u = 1/2 and the expected return is 1/4.
In period t < N, if the blackmailer decide to demand a payment of ut, the probability that he can keep
blackmailing is 1 − u and he can get u plus the future return, Vt+1(ut+1). So, the value function is
�
�
Vt(u) = max R, max {(1 − u)[u + Vt+1(ut+1)]}
(2)
= max
= max
�
u∈[0,1]
�
R, max
u∈[0,1]
� �
R, max
u∈[0,1]
−
−
�
u −
�
u −
We can analyze this problem according to the value of R.
First, observe that in period N − 1, if R ≥ 1/4, we have
� ��
2
2
(1 − Vt+1)
(1 − Vt+1)
+ Vt+1 +
2
4
�2 (1 − Vt+1)
+
2
(1 + Vt+1) 2
��
4
VN−1(u) = max{R, max {(1 − u)(u + R)}}
u∈[0,1]
� � �
� ��
2
(1 − R) (1 − R)2
= max R, max − u − + R +
u∈[0,1]
2
4
= R +
(1 − R)2
4
In this case, if R < 1, demanding a payment u is better than accepting a lump sum and the optimal uN−1
will be (1 − R)/2. Also, we have VN−1 = (1+R)2
< 1. Then in the previous period, from equation (2), u will be
(1−VN−1)
2
(1+VN−1) 2
4
and VN−2 = 4 < 1. Hence, we know that even though the Value function is nondecreasing
in −t, it is bounded above by one. This tell us that the blackmailer will always choose to demand a payment
.
ut = (1+VN−1)
2
However, if R ≥ 1, the optimal u is 0 because u ≥ 0 and the last term in the maximization is concave in u.
The blackmailer gets the same return no matter what his decision is. However, we know that waiting for one
period doesn’t give the blackmailer any benefit. Hence, we know if R > 1, the blackmailer will accept a lump
sum R in the first period. If R < 1/4, we have
� � �
VN−1(u) = max R, max (1 − u) u +
u∈[0,1]
1
���
4
� � �
= max R, max − u −
u∈[0,1]
3
�2 +
8
1
��
9
+
4 64
= 1 9
+
4 64
Again, demanding a payment u is better than accepting a lump sum. The optimal uN−1 will be 3/8. This is
just the same as the case that R = 1/4.
In conclusion, we have
1. If R ≥ 1, the optimal policy is accept a lump sum R in the first period.
2
(1)
(3)
(4)

2. If 1 > R ≤ 1/4, the optimal policy is demanding a payment uk = (1+Vk+1)
2 when k < N. And, in the last
period, accept R instead of require a payment u.
3. If R < 1/4, the optimal policy is always demanding a payment uk = (1+Vk+1)
2 while the last u should be
1/2 to maximize the whole profit.
3