Evolution and Optimum Seeking

Multidimensional Strategies 41
ascheme of some sort, not always in an optimal way under the assumption of a speci ed
internal model. Thus the risk is run of not being able to improve the objective function
value at each step. Failures must accordingly be planned for, if something can also be
\learned" from them. This trial character of search strategies has earned them the name
of trial-and-error methods. The most important of them that are still in current use will
be presented in the following chapters. Their attraction lies not in theoretical proofs of
convergence and rates of convergence, but in their simplicity and the fact that they have
proved themselves in practice. In the case of convex or quadratic unimodal objective
functions, however, they are generally inferior to the rst and second order strategies to
be described later.
3.2.1.1 Coordinate Strategy
The oldest of multidimensional search procedures trades under a variety of names (e.g.,
successive variation of the variables, relaxation, parallel axis search, univariate or univariant
search, one-variable-at-a-time method, axial iteration technique, cyclic coordinate
ascent method, alternating variable search, sectioning method, Gauss-Seidel strategy) and
manifests itself in a large number of variations.
The basic idea of the coordinate strategy, as it will be called here, comes from linear
algebra and was rst put into practice by Gauss and Seidel in the single step relaxation
method of solving systems of linear equations (see Ortega and Rocko , 1966 Ortega and
Rheinboldt, 1967 VanNorton, 1967 Schwarz, Rutishauser, and Stiefel, 1968). As an
optimization strategy it is attributed to Southwell (1940, 1946) or Friedmann and Savage
(1947) (see also D'Esopo, 1959 Zangwill, 1969 Zadeh, 1970 Schechter, 1970).
The parameters in the iteration formula (3.20) are varied in turn individually, i.e., the
search directions are xed by the rule:
v (k) = e` with ` =
( n if k = pn p integer
k (mod n) otherwise
where e` is the unit vector whose components have thevalue zero for all i 6= `, and unity
for i = `. In its simplest form the coordinate strategy uses a constant steplengths (k) .
Since, however, the direction to the minimum is unknown, both positive and negative
values of s (k) must be tried. In a rst and easy improvement on the basic procedure, a
successful step is followed by further steps in the same direction, until a worsening of the
objective function is noted. It is clear that the choice of step length strongly in uences
the number of trials required on the one hand and the accuracy that can be achieved in
the approximation on the other.
One can avoid the problem of the choice of step length most e ectively by using a line
search method each time to locate the relative optimum in the chosen direction. Besides
the interval division methods, the Fibonacci search and the golden section, Lagrangian
interpolation can also be used, since all these procedures work without knowledge of the
partial derivatives of the objective function. A further strategy for boxing in the minimum
must be added, in order to establish a suitable starting interval for each one dimensional
minimization.