STOCHASTIC

(b) Show that the optimal policy in (i) reduces to dollar averaging when a — b.
(c) Interpret the policy in (i) when b (the maximum possible price decrease) is very small.
(d) Derive an optimal policy similar to (i) which would be appropriate when a (the
maximum possible price increase) is small.
14. Referring to Pye's paper in Chapter 2 of this part, let p, (1 g t g T) be the asset prices,
and suppose the p, follow an arithmetic random walk: p, = Pt-i+yt, t= 1,2,.... The y,
are independent and identically distributed. Let a and b be the maximum possible price
increases and decreases: — b g y, g a. Assume that bit a. Consider the case of a nondivisible
asset, where the only possible decisions are to sell or not to sell the entire asset. Let
pt* = maxosisi(^i), Rt = P*~Pt be the regret at time t, and let P0 be given.
(a) Show that R, and pt* obey the recursion relations
R, + 1 = max(0,/J,-^, + i), p?+i = p,* + max(0,-i?,+j,+1).
Let/„(i?) be the maximum regret given R, given that the asset has not been sold with n
periods to go, and given that a minimax policy is used in the future. Note that/0(i?) = R.
(b) Show that
Mr)- min {max (/{,«), R+b] = max(R,a),
(sell) (retain)
and the minimax policy is to sell on the next-to-last period for any R.
(c) Show by induction that
/„ (R) = min {max(R + jb, (n -j) a)}.
OSJSn-l
(d) Show that an optimal stopping rule is to sell the asset with n periods to go if the
current value of R satisfies R^na — b, and to wait one more period before deciding if
R < na — b.
(e) If ka g b 5j (k+ 1)«, show that the asset will always be sold at least k periods prior
to the terminal date.
(f) If i?i = 0, show that the asset will not be sold prior to
where [x] denotes the largest integer gjc.
a + b
b
a+b
15. Consider a three-state Markov chain with transition matrix
"i i i
P= i 0 i
.4 i 0
Refer to Exercises ME-15 and 16 for notation and concepts.
(a) Find the limiting distribution n,, i = 1,2,3.
(b) Show that the mean first passage times ptj are
2 2 2
[ful =
4 1
4 4
where fiu is the expected recurrence time of state ;'.
Suppose that a cost CO) is incurred each time the chain is in state *': C(l) = 2, C(2) = 0,
CO) = 1.
670 PART V DYNAMIC MODELS