guidance, flight mechanics and trajectory optimization

There are several ways to accomplish the specified objective and at the
same time minimize the performance index. The most direct approach would
involve evaluating the performance index for every possible set of decisions.
However, in most decision processes the number of different decision sets is
so large that such an evaluation is computationally impossible. A second
approach would be to endow the problem with a certain mathematical structure
( e.g., continuity, differentiability, analyticity, etc.), and then use a
standard mathematical technique to determine certain additional properties
which the optimal decision sequence must have. Two such mathematical
techniques are the maxima-minima theory of the Differential Calculus and
the Calculus of Variations. A third alternative is to use Dynamic Programming.
Dynamic Programming is essentially a systematic search procedure for
finding the optimal decision sequence; in using the technique it is only
necessary to evaluate the performance index associated with a small number
of all possible decision sets. This approach differs from the well-known
variational methods, in that it is computational in nature and goes directly
to the determination of the optimal decision sequence without attempting to
uncover any special properties which this decision sequence might have. In
this sense the restrictions on the problem's mathematical structure, which
are needed in the variational approach, are totally unnecessary in Dynamic
Programming. Furthermore, the inclusion of constraints in the problem, a
situation which invariably complicates a solution of the variational methods,
facilitates solution generation in the Dynamic Programming approach since
the constraints reduce the number of decision sets over which the search
must be conducted.
The physical basis for Dynamic Programming lies in the "Principle of
Optimality," a principle so simple and so self -evident that one would
hardly expect it could be of any importance. However, it is the recognition
of the utility of this principle along with its application to a broad
spectrum of problems which constitutes Bellman's major contribution.
Besides its value as a computational tool, Dynamic Progr arming is also
of considerable theoretical importance. If the problem possesses a certain
mathematical structure, for example, if it is describable by a system of
differential equations, then the additional properties of the optimal
decision sequence, as developed by the Maximum Principle or the Calculus
of Variations, can also be developed using Dynamic Programming. This feature
gives a degree of completeness to the area of multi-stage decision processes
and allows the examination of problems from several points of view. Further-
more, there is a class of problems, namely stochastic decision processes,
which appear to lie in the variational domain, and yet which escape analysis
by means of the Variational Calculus or the Maximum Principle. As will be
shown, it is a rather straightforward matter to develop the additional
properties of the optimal stochastic decision sequence by using Dynamic
Programming.
5