Evolution and Optimum Seeking

Particular Problems and Methods of Solution 15
Subsidiary conditions, or constraints, complicate matters. In rare cases equality constraints
can be expressed as equations in one variable, that can be eliminated from the
objective function, or constraints in the form of inequalities can be made super uous
by a transformation of the variables. Otherwise there are the methods of bounded variation
and Lagrange multipliers, in addition to penalty functions and the procedures of
mathematical programming.
The situation is very similar for functional optimization, except that here the indirect
methods are still dominanteven today. Thevariational calculus provides as conditions for
optima di erential instead of ordinary equations{actually ordinary di erential equations
(Euler-Lagrange) or partial di erential equations (Hamilton-Jacobi). In only a few cases
can such a system be solved in a straightforward way for the unknown functions. One
must usually resort again to the help of a computer. Whether it is advantageous to
use a digital or an analogue computer depends on the problem. It is a matter of speed
versus accuracy. Ahybrid system often turns out to be especially useful. If, however, the
solution cannot be found by a purely analytic route, why not choose from the start the
direct procedure also for functional optimization? In fact with the increasing complexity
of practical problems in numerical optimization, this eld is becoming more important, as
illustrated by the work of Daniel (1969), who takes over methods without derivatives from
parameter optimization and applies them to the optimization of functionals. An important
point in this is the discretization or parameterization of the originally continuous problem,
which canbeachieved in at least two ways:
By approximation of the desired functions using a sum of suitable known functions or
polynomials, so that only the coe cients of these remain to be determined (Sirisena,
1973)
By approximation of the desired functions using step functions or sides of polygons,
so that only heights and positions of the connecting points remain to be determined
Recasting a functional into a parameter optimization problem has the great advantage
that a digital computer can be used straightaway to nd the solution numerically. The
disadvantage that the result only represents a suboptimum is often not serious in practice,
because the assumed values of parameters of the process are themselves not exactly
known (Dixon, 1972a). The experimentally determined numbers are prone to errors or
to statistical uncertainties. In any case, large and complicated functional optimization
problems cannot be completely solved by the indirect route.
The direct procedure can either start directly with the functional to be minimized, if
the integration over the substituted function can be carried out (Rayleigh-Ritz method)
or with the necessary conditions, the di erential equations, which specify the optimum. In
the latter case the integral is replaced by a nite sum of terms (Beveridge and Schechter,
1970). In this situation gradient methods are readily applied (Kelley, 1962 Klessig and
Polak, 1973). The detailed way to proceed depends very much on the subsidiary conditions
or constraints of the problem.