STOCHASTIC

(a) Show that/, (b, c) = gt (Z>, c).
(b) Develop the functional equation
f„(b,c) = ma.x0SXllib max0£ynSc[g„(x„,y„) + fn+ t(b-x„,c-y„)] for n = 2,3,....M
(c) Develop a grid algorithm to solve the problem in (b).
(d) How much more complicated is the two-dimensional case as opposed to the onedimensional
case considered in Exercise 20 ?
(e) Apply the algorithm when N=3, 6 = 10, c = 15, gi(xuyi) = x^+yS,
g2(x2,y2) = 4x1 2 +y2, and g3(x3,y3) = 4x2 2 .
Consider the problem of maximizing the modified function
ffi(.x1,y1)+ ••• +gn(xN,yN)- X[yi + ---+yN]
s.t. xi + ••• + xN — by (*)
and all xt ,yi £ 0, where A is a fixed parameter. Assume that
lim),„-.»[s'nUn,j'n)/>'»] = 0.
(f) Show that the maximization over y„ and x„ may be done independently.
(g) Show that
rn(x„,K) = h„(x„) = max[gn(x„,y„)-Xy„].
(h) Show that the original problem is then equivalent to maximizing htix^-i \-hs (xN)
subject to (*) if X is varied until the restriction £ )".W) = c is met.
(i) Modify the algortihm in Exercise 20 to solve this problem.
22. Suppose an economy has a productive capcaity of M in year 1 and we wish to plan
production for the years 1,...,N. Productive capacity can be used either for the production
of consumer goods or for the development of additional production capacity. Consumers
receive f(x) dollars worth of goods if x units of the productive capacity are allocated (for
one year) to the manufacture of consumer goods. At the end of the year, some of the
productive capacity allocated to the production of consumer goods will be worn out, and
only a quantity ax, 0 < a < 1, will be usable in the following year. If y units of productive
capacity are allocated to the formation of capital goods, then at the end of the year, we have
a productive capacity of Sy, 6 > 1, corresponding to these y units. Suppose that a, 5, and /
are stationary in time but that consumer goods obtained at a later time are not as valuable
as those available immediately. The one-period discount factor is 0 < r < 1. It is of interest
to determine how the economy should divide its productive capacity between consumption
and investment goods in each of the N years so as to maximize the present value of the
economy's consumption.
(a) Formulate the problem as a dynamic program.
(b) Develop a flow chart indicating the calculations involved to solve the problem
for general/
Suppose/(x) = a + bx, where a,b > 0.
(c) Find the optimal policy.
(d) Let M = 10, N = 5, r = 0.9, a = i, 5 = 1$, a = 1, and b = }. Find the optimal
decisions.
Suppose now that 0andpr{3==32} = 1—Pi-
(e) Formulate a dynamic programming model of the economy's problem assuming that
the goal is to maximize the present value of the economy's expected consumption.
(f) Let Pi = p2 = |,