guidance, flight mechanics and trajectory optimization

The extension of Dynamic Programmin g to these other fields, however,
presented computational problems. For example, it became necessary to
study topics such as accuracy, stability and storage in order to handle
these more complicated problems. One of the beauties of Dynamic Programming
came to rescue in solving some of these problems. It is idiosyncrasy
exploitation. Whereas problem peculiarities usually are a burden to classical
techniques, they are usually blessings to the dynamic programmer. It is
possible to save computation time, to save storage and/or to increase accuracy
by exploiting problem peculiarities in Dynamic Programming.
An understanding of Dynamic Programming hinges on an understanding of
the concept of a multi-stage decision process, a concept which is most
easily described by means of an example. Consider a skier at the top of
a hill who wishes to get down to the bottom of the hill as quickly as
possible. Assume that there are several trails available which lead to
the bottom and that these trails intersect and criss-cross one another as
the slope is descended. The down hill path which is taken will depend only
on a sequence of decisions which the skier makes. The first decision consists
of selecting the trail on which to start the run. Each subsequent decision
is made whenever the current trail intersects some new trail, at which point
the skier must decide whether to take the new trail or not. Thus, associated
with each set of decisions is a path leading to the bottom of the hill, and
associated with each path is a time, namely the time it takes to negotiate
the hill. The problem confronting the skier is that of selecting that
sequence of decisions (i.e., the particular combination of trails) which
result in a minimum run time.
From this example, it is clear that a multi-stage decision process
possesses three important features:
(1) To accomplish the objective of the process (in the example
above, to reach the bottom of the hill) a sequence of
decisions must be made.
(2) The decisions are coupled in the sense that the ntJ decision is
affected by all the prior decisions, and it, in turn, effects
all the subsequent decisions. In the skier example, the very
existence of an n4fi decision depends on the preceding decisions.
(3) Associated with each set of decisions there is a number which
depends on all the decisions in the set (e.g., the time to
reach the bottom of the hill). This number which goes by a
variety of names will be referred to here as the performance
index. The problem is to select that set of decisions which
minimizes the performance index.