guidance, flight mechanics and trajectory optimization

2.4 LIMITING PROCESS IN DYNAMIC PROGRAMMING
The previous sections have delt exclusively with the computational
aspects of Dynamic Programing and have shown how the Principle of Optima-
lity can be used to systematize the search procedure for finding an optimal
decision sequence. As mentioned in Section 2.1, Dynamic Programming is
also a valuable theoretical tool in that it can be used to develop additional
properties of the optimal decision sequence. For example, it is well known
that the optimal solution for the problem of Lagrange (Section 2.2.2) must
satisfy the Euler-Lagrange equation. This differential equation, as well
as other conditions resulting from either an application of the classical
Calculus of Variations or the Pontryagin Maximum Principle, can also be
developed through Dynamic Programming.
To develop these additional properties, the multi-stage decision
process must be considered in the limit as the separation between neigh-
boring states and decisions go to zero (i.e., as the process becomes continuous).
That is, the problem is first discretized and a finite number of states and
decisions considered just as in the computational approach of the previous
sections. The Principle of Optimality is then used to develop a recursive
equation by which the numerical values of the optimal decision sequence
are computed. (Th is equation was not given an explicit statement in the
previous sections since it was reasonably obvious there how the Principle
of Optimality was to be used in the search process.) By considering the
discretized process in the limit (i.e., allowing it to become a continuous
process again), the recursive equation which governs the search procedure
in the discrete case becomes a first-order , partial differential equation.
From this partial differential equation, many additional properties of the
optimal decision sequence can be developed.
It should be mentioned that in some cases the limiting process outlined
does not exist and the passage to the limit leads to an erroneous result.
While this situation does occur in physically meaningful problems and, there-
fore, cannot be classed as pathological, it occurs infrequently enough as
to cause little concern. Some examples of this phenomenon will be given
later on.
2.4.1 Recursive Equation for the Problem of Lagrange
Consider the one-dimensional Lagrange problem of minimizing the
integral
61
(2.4.1)