Evolution and Optimum Seeking

Particular Problems and Methods of Solution 11
themselves functions of one or more parameters, the objective function is a function of a
function, or a functional.
A classical problem is to determine the smooth curve down which apoint mass will
slide between two points in the shortest time, acted upon by the force of gravity and
without friction. Known as the brachistochrone problem, it can be solved by means of the
ordinary variational calculus (Courant and Hilbert, 1968a,b Denn, 1969 Clegg, 1970). If
the functions to be determined depend on several variables it is a multidimensional variational
problem (Klotzler, 1970). In many cases the time t appears as the only parameter.
The objective function is commonly an integral, in the integrand of which will appear not
only the independent variables
x(t) =fx 1(t)x 2(t):::xn(t)g
but also their derivatives _xi(t) =@xi=@t and sometimes also the parameter t itself:
F (x(t)) =
Z t2
t1
f(x(t) _x(t)t) dt ! extr
Such problems are typical in control theory, where one has to nd optimal controlling
functions for control processes (e.g., Chang, 1961 Lee, 1964 Leitmann, 1964 Hestenes,
1966 Balakrishnan and Neustadt, 1967 Karreman, 1968 Demyanov and Rubinov, 1970).
Whereas the variational calculus and its extensions provide the mathematical basis
of functional optimization (in the language of control engineering: optimization with distributed
parameters), parameter optimization (with localized parameters) is based on the
theory of maxima and minima from the elementary di erential calculus. Consequently
both branches have followed independent paths of development and become almost separate
disciplines. The functional analysis theory of Dubovitskii and Milyutin (see Girsanov,
1972) has bridged the gap between the problems by allowing them to be treated as special
cases of one fundamental problem, and it could thus lead to a general theory of
optimization. However di erent their theoretical bases, in cases of practical signi cance
the problems must be solved on a computer, and the iterative methods employed are then
broadly the same.
One of these iterative methods is the dynamic programming or stepwise optimization
of Bellman (1967). It was originally conceived for the solution of economic problems, in
which time-dependent variables are changed in a stepwise way at xed points in time.
The method is a discrete form of functional optimization in which the trajectory sought
appears as a steplike function. At each step a decision is taken, the sequence of which is
called a policy. Assuming that the state at a given step depends only on the decision at
that step and on the preceding state{i.e., there is no feedback{, then dynamic programming
can be applied. The Bellman optimum principle implies that each piece of the optimal
trajectory that includes the end point is also optimal. Thus one begins by optimizing the
nal decision at the transition from the last-but-one to the last step. Nowadays dynamic
programming is frequently applied to solving discrete problems of optimal control and
regulation (Gessner and Spremann, 1972 Lerner and Rosenman, 1973). Its advantage
compared to other, analytic methods is that its algorithm can be formulated as a program
suitable for digital computers, allowing fairly large problems to be tackled (Gessner and