Evolution and Optimum Seeking

Particular Problems and Methods of Solution 13
2.2.4 Direct (Numerical) Versus
Indirect (Analytic) Optimization
The classi cation of mathematical methods of optimization into direct and indirect procedures
is attributed to Edelbaum (1962). Especially if one has a computer model of a
system, with which one can perform simulation experiments, the search for a certain set
of exogenous parameters to generate excellent results asks for robust direct optimization
methods. Direct or numerical methods are those that approach the solution in a stepwise
manner (iteratively), at each step (hopefully) improving the value of the objective
function. If this cannot be guaranteed, a trial and error process results.
An indirect or analytic procedure attempts to reach the optimum in a single (calculation)
step, without tests or trials. It is based on the analysis of the special properties of
the objective function at the position of the extremum. In the simplest case, parameter
optimization without constraints, one proceeds on the assumption that the tangent plane
at the optimum is horizontal, i.e., the rst partial derivatives of the objective function
exist and vanish in x :
@F
=0 for all i = 1(1)n (2.1)
@xi x=x
This system of equations can be expressed with the so-called Nabla operator (r) asa
single vector equation for the stationary point x :
rF (x )=0 (2.2)
Equation (2.1) or (2.2) transforms the original optimization problem into a problem of
solving a set of, perhaps non-linear, simultaneous equations. If F (x) or one or more of its
derivatives are not continuous, there may be extrema that do not satisfy the otherwise
necessary conditions. On the other hand not every point inIR n {the n-dimensional space
of real variables{ that satis es conditions (2.1) need be a minimum it could also be a
maximum or a saddle point. Equation (2.2) is referred to as a necessary condition for the
existence of a local minimum.
To givesu cient conditions requires further processes of di erentiation. In fact,
di erentiations must be carried out until the determinant of the matrix of the second
or higher partial derivatives at the point x is non-zero. Things remain simple in the case
of only one variable, when it is required that the lowest order non-vanishing derivative is
positive and of even order. Then and only then is there a minimum. If the derivative is
negative, x represents a maximum. A saddle point exists if the order is odd.
For n variables, at least the n
(n +1) second partial derivatives
2
@2F (x)
for all i j = 1(1)n
@xi @xj
must exist at the point x . The determinant of the Hessian matrix r 2 F (x )must be
positive, as well as the further n ; 1 principle subdeterminants of this matrix. While
MacLaurin had already completely formulated the su cient conditions for the existence
of minima and maxima of one parameter functions in 1742, the corresponding theory