Evolution and Optimum Seeking

Particular Problems and Methods of Solution 17
reset into the allowed region by thecomplex method of M. J. Box (1965) (a direct search
strategy) whenever explicit bounds are crossed. Implicit constraints on the other hand
are treated as barriers (see Chap. 3, Sect. 3.2.1.6).
The methods of mathematical programming, both linear and non-linear, treat the
constraints as the main aspect of the problem. They were specially evolved for operations
research (Muller-Merbach, 1971) and assume that all variables must always be positive.
Such non-negativity conditions allow special solution procedures to be developed. The
simplest models of economic processes are linear. There are often no better ones available.
For this purpose Dantzig (1966) developed the simplex method of linear programming (see
also Krelle and Kunzi, 1958 Hadley, 1962 Weber, 1972).
The linear constraints, together with the condition on the signs of the variables, span
the feasible region in the form of a polygon (for n = 2) or a polyhedron, sometimes called
simplex. Since the objective function is also linear, except in special cases, the desired
extremum must lie in a corner of the polyhedron. It is therefore su cient just to examine
the corners. The simplex method of Dantzig does this in a particularly economic way, since
only those corners are considered in which the objective function has progressively better
values. It can even be thought of as a gradient method along the edges of the polyhedron.
It can be applied in a straightforward way to manyhundreds, even thousands, of variables
and constraints. For very large problems, which mayhave a particular structure, special
methods have also been developed (Kunzi and Tan, 1966 Kunzi, 1967). Into this category
come the revised and the dual simplex methods, the multiphase and duplex methods, and
decomposition algorithms. An unpleasant property of linear programs is that sometimes
just small changes of the coe cients in the objective function or the constraints can cause
a big alteration in the solution. To reveal such dependencies, methods of parametric linear
programming and sensitivity analysis have been developed (Dinkelbach, 1969).
Most strategies of non-linear programming resemble the simplex method or use it
as a subprogram (Abadie, 1972). This is the case in particular for the techniques of
quadratic programming, which are conceived for quadratic objective functions and linear
constraints. The theory of non-linear programming is based on the optimality conditions
developed by Kuhn and Tucker (1951), an extension of the theory of maxima and minima
to problems with constraints in the form of inequalities. These can be expressed
geometrically as follows: at the optimum (in a corner of the allowed region) the gradient
of the objective function lies within the cone formed by the gradients of the active
constraints. To start with, this is only a necessary condition. It becomes su cient under
certain assumptions concerning the structure of the objective and constraint functions.
For minimum problems, the objective function and the feasible region must be convex,
that is the constraints must be concave. Such a problem is also called a convex program.
Finally the Kuhn-Tucker theorem transforms a convex program into an equivalent saddle
point problem (Arrow and Hurwicz, 1956), just as the Lagrange multiplier method does
for constraints in the form of equalities. A complete theory of equality constraints is due
to Apostol (1957).
Non-linear programming is therefore only applicable to convex optimization, in which,
to be precise, one must distinguish at least seven types of convexity (Ponstein, 1967). In
addition, all the functions are usually required to be continuously di erentiable, with an