Evolution and Optimum Seeking

Introduction 3
simple matter to collect several thousand published articles about optimization methods.
Even an interested party nds it di cult to keep pace nowadays with the development
that is going on. It seems far from being over, for there still exists no all-embracing theory
of optimization, nor is there any universal method of solution. Thus it is appropriate, in
Chapter 2, to give a general survey of optimization problems and methods. The special
r^ole of direct, static, non-discrete, and non-stochastic parameter optimization emerges
here, for many of these methods can be transferred to other elds the converse is less often
possible. In Chapter 3, some of these strategies are presented in more depth, principally
those that extract the information they require only from values of the objective function,
thatistosay without recourse to analytic partial derivatives (derivative-free methods).
Methods of a probabilistic nature are omitted here.
Methods which use chance as an aid to decision making, are treated separately in
Chapter 4. In numerical optimization, chance is simulated deterministically by means of
a pseudorandom number generator able to produce some kind of deterministic chaos only.
One of the random strategies proves to be extremely promising. It imitates, in a highly
simpli ed manner, the mutation-selection game of nature. This concept, a two membered
evolution strategy, is formulated in a manner suitable for numerical optimization in Chapter
5, Section 5.1. Following the hypothesis put forward by Rechenberg, that biological
evolution is, or possesses, an especially advantageous optimization process and is therefore
worthy of imitation, an extended multimembered scheme that imitates the population
principle of evolution is introduced in Chapter 5, Section 5.2. It permits a more natural
as well as more e ective speci cation of the step lengths than the two membered scheme
and actually invites the addition of further evolutionary principles, such as, for example,
sexual propagation and recombination. An approximate theory of the rate of convergence
can also be set up for the (1 , )evolution strategy, inwhich only the best of descendants
of a generation become parents of the following one.
A short excursion, new to this edition, introduces nearly concurrent developments
that the author was unaware of when compiling his dissertation in the early 1970s, i.e.,
genetic algorithms, simulated annealing, andtabu search.
Chapter 6 then makes a comparison of the evolution strategies with the direct search
methods of zero, rst, and second order, which were treated in detail in Chapter 3.
Since the predictive power of theoretical proofs of convergence and statements of rates
of convergence is limited to simple problem structures, the comparison includes mainly
numerical tests employing various model objective functions. The results are evaluated
from two points of view:
E ciency, or speed of approach to the objective
E ectivity, or reliability under varying conditions
The evolution strategies are highly successful in the test of e ectivity orrobustness.
Contrary to the widely held opinion that biological evolution is a very wasteful method
of optimization, the convergence rate test shows that, in this respect too, the evolution
methods can hold their own and are sometimes even more e cient than many purely
deterministic methods. The circle is closed in Chapter 7, where the analogy between