STOCHASTIC

costs, and transfer costs having both fixed and proportional components. The
possible cash balance states were assumed to be discrete; and withdrawals or
deposits were assumed to be a sequence of discrete iid random variables. The
objective was minimization of discounted expected cost. For any given policy,
changes in the cash balance thus become a Markov chain, and the cash balance
problem becomes a standard Markovian decision problem. As noted by Manne
(1960), such problems can be reformulated in a linear programming framework.
See Exercise I-ME-20 for a linear programming approach to general Markovian
decision problems, and Exercise ME-8 for applications to bond-option
strategies. A common drawback of the standard linear programming formulation
is the large size of the resulting problem. As noted by de Ghellinck and
Eppen (1967), significant reductions in problem size obtain for the special but
important class of so-called separable Markovian decision problems. In
separable problems, the cost of making a decision k when in state i reduces to
a sum of a function of k and a function of /; and the transition probabilities
given decision k and state i are independent of *'. The properties hold for the
cash balance problem. First, costs are clearly additive in the states and decisions.
Second, the firm decides to start the current period in a particular state, so the
transition probabilities are actually independent of the previous period's
ending state. Exercise ME-9 presents the de Ghellinck and Eppen formulation
for linear programming in discounted Markovian decision problems, and
gives another proof of the optimality of stationary policies. Exercise ME-10
outlines their theory of linear programming for separable Markovian decision
problems, in the special case of the cash balance problem. It was this formulation
of the problem that Eppen and Fama (1968) used in their study. They
solved a large number of examples numerically, and found that, in all cases,
the optimal policy had the following simple (u, U; D, d) form. Move the cash
balance down to D when it exceeds a control limit d, move it up to U when
it becomes less than a control limit w, and do nothing otherwise.
In a recent paper, Eppen and Fama (1969) studied the cash balance problem
using discounted stochastic dynamic programming. The basic problem
formulation was f hat of their earlier 1968 paper, except that fixed components
of transfer costs were assumed to be absent. By starting with a finite horizon
problem and then going to the infinite horizon limit, they were able to show
that the optimal policy was of the following simple (U,D) form: Move the
cash balance down to D whenever it exceeds D, move it up to U when it
becomes less than U, and do nothing otherwise. Their proof of this result is
presented in Exercise ME-11. The form of the optimal policy with fixed
transfer costs included appears to be somewhat uncertain at this time. Neave
(1970) has studied this problem, and has found that the optimal policy is either
of the simple (u, U; D,d) form, or else is of the form (u, U,u + ;d~,D,d):
Move the cash balance down to D or up to U whenever it is greater than d
446 PART V DYNAMIC MODELS