STOCHASTIC

19. Refer to the "Introduction to Dynamic Programming." Verify that the following
return functions satisfy the monotonicity assumption.
(a) vd(x) = r(x, [z:x,^;[,«]us(z)}, where p(-)S 0.
(e) When would one obtain these return functions?
(f) Show that (a)-(d) may be generalized by replacing " + " and multiplication wherever
they appear by ffi, where ffi is any commutative, symmetric binary operator that preserves
inequalities on a subset H of the real numbers; that is, (a © b) ® c = a © (6 © c),
a © b = b © a, and if a £ b, then a © c S 6 © c.
(g) Verify that the following are © operators:
a © b = a + b, a © b = max {a, 6}, a ffi 6 = min{a,6};
a ffi b = e a+b and a ® b - ab,
the last if i/ is the nonnegative orthant.
(h) Show that the monotonicity assumption is satisfied in the consumption-investment
model if the «t are monotone nondecreasing.
(i) Suppose «,(ct) = «, + litCt + y,ct 2 + y,ct 3 . Determine conditions on the parameters
at,A,7t, and 6, so that the monotonicity assumption is satisfied for all c, e [0, Af],
where M < co.
20. Consider the allocation problem
/v(6) = max R (x i,..., xN) = gi(xl)+ h #*(**)
s.t. xi + --- + Xs = b, all x„ g 0.
Hence fN (b) is the optimal return from an allocation of resources b to A'activities.
(a) Derive the functional equation
f„(b) = max0SXiiSb{g„(xn)+f„-i(b-x„)} for n = 2,3,...,A r and b g 0.
(b) Interpret the meaning of the functional equation in (a).
The recurrence relation in (a) yields a theoretical method for obtaining the sequence
{fn(b)} inductively once fi(6) is determined.
(c) Show that/!(/>)=#! (6).
(d) Develop an algorithm to solve the functional equation in (a) based on the grid
0,A,2A,...,rA = 6.
(e) Apply the algorithm in (d) when N = 3, b = 10, gx(xt) = x^, g2(x2) = 4x2 2 , and
g3(x3) = 8x3.
Suppose now that the x„ are subject to the constraints a„ S x„ S b„.
(f) Derive the new functional equation.
(g) Modity the algorithm in (d) to solve the functional equation in (f).
(h) Solve the problem in (e) when (01,^2,03) = (0,1,0) and (61,62,63) = (1,2,4).
21. This problem is a generalization of Exercise 20 to the two-dimensional constraint
case and illustrates the use of Lagrange multipliers in dynamic programming problems.
Consider the two-dimensional resource allocation problem.
fN(b,c) = maxR(xu...,xN,y1,...,yN) = £ gt(xt,y,),
s.t. £•*» =b, all xi g 0,
X>< = c, yt £ 0.
62 PART I MATHEMATICAL TOOLS
N