Evolution and Optimum Seeking

160 Evolution Strategies for Numerical Optimization
shown, GAs other than ESs favor in-breadth search andthus are especially prepared to
solve global and discrete optimization problems, where a volume-oriented approach is
more appropriate than a path-oriented one. They have so far done their best in all kinds
of combinatorial optimization (e.g., Lawler et al., 1985), a eld that has not been pursued
in depth throughout this book. One example in the domain of computational intelligence
has been the combined topology and parameter optimization of arti cial neural networks
(e.g., Mandischer, 1993) another is the optimization of membership function parameters
within fuzzy controllers (e.g., Meredith, Karr, and Kumar, 1992).
5.4 Simulated Annealing
The simulated annealing approach tosolveoptimization problems does not really belong
to the biologically motivated evolutionary algorithms. However, it belongs to the realm of
problem solving methods that make use of other natural paradigms. This is the reason why
this section has not been placed elsewhere among the traditional hill climbing strategies.
In order to harden steel one rst heats it up to a high temperature not far away
from the transition to its liquid phase. Subsequently one cools down the steel more or
less rapidly. This process is known as annealing. According to the cooling schedule the
atoms or molecules have more or less time to nd positions in an ordered pattern (e.g.,
a crystal structure). The highest order, which corresponds to a global minimum of the
free energy, canbeachieved only when the cooling proceeds slowly enough. Otherwise
the frozen status will be characterized by one or the other local energy minimum only.
Similar phenomena arise in all kinds of phase transitions from gaseous to liquid and from
liquid to solid states.
A descriptive mathematical model abstracts from local particle-to-particle interactions.
It describes statistically the correspondences between macro variables like density,
temperature, and entropy. It was Boltzmann who rst formulated a probability lawto
link the temperature with the relative frequencies of the very many possible micro states.
Metropolis et al. (1953) simulated on that basis the evolution of a solid in a heat bath
towards thermal equilibrium. By means of a Monte-Carlo method new particle con gurations
were generated. Their free energy Enew was compared with that of the former
state (Eold). If Enew Eold then the new con guration \survives" and forms the basis
for the next perturbation. The new state may survivealsoifEnew >Eold, but only with
a certain probability w
w = 1
c exp Eold ; Enew
KT
where K denotes the famous Boltzmann constant andTthe current temperature. The
constant c serves to normalize the probability distribution. This Metropolis algorithm
thus is in line with the probability lawof Boltzmann.
Kirkpatrick, Gelatt, and Vecchi (1983) and Cerny (1985) published optimization methods
based on Metropolis' simulation algorithm. These methods are used quite frequently
nowadays as simulated annealing (SA) procedures. Due to the fact that good intermediate
positions may be \forgotten" during the searchforaminimum or maximum, the algorithm
is able to escape from local extrema and nally might reach the global optimum.