Evolution and Optimum Seeking

108 Evolution Strategies for Numerical Optimization
The question remains of how tochoose the random vectors z (g) .Thischoice has the
r^ole of mutation. Mutations are understood nowadays to be random, purposeless events,
which furthermore only occur very rarely. Ifoneinterprets them, as is done here, as a sum
of many individual events, it is natural to choose a probability distribution according to
which small changes occur frequently, but large ones only rarely (the central limit theorem
of statistics). For discrete variations one can use a binomial distribution, for example, for
continuous variations a Gaussian or normal distribution.
Two requirements then arise together by analogy with natural evolution:
That the expectation value i for a component zi has the value zero
2
That the variance i ,theaverage squared deviation from the mean, is small
The probability density function for normally distributed random events is (e.g., Heinhold
and Gaede, 1972):
w(zi) =
1
p
2 i
exp
!
(5.1)
; (zi ; i) 2
2 2 i
2
If i = 0, one obtains a so-called (0 i ) normal distribution. There are still however a
total of n free parameters f i i= 1(1)ng with which to specify the standard deviations
of the individual random components. By analogy with other, deterministic search strategies,
the i can be called step lengths, in the sense that they represent average values of
the lengths of the random steps.
For the occurrence of a particular random vector z = fzi i = 1(1)ng, with the
2
independent (0 i ) distributed components zi, the probability density function is
w(z1z2:::zn) =
nY
i=1
w(zi) =
(2 ) n 2
or more compactly, if i = for all i =1(1)n,
w(z) =
1
p2
! n
1
nQ
i=1
exp
i
exp
;zz T
2 2
!
; 1
2
nX
i=1
zi
i
2 !
(5.2)
(5.3)
q Pn
i=1 z 2 i a p 2 distribution is ob-
For the length of the overall random vector S =
2 tained. The distribution with n degrees of freedom approximates, for large n, to
a( q n ; 1
2 2
) normal distribution. Thus the expectation value for the total length
2
of the random vector for many variables is E(S) = p n,thevariance is D2 (S) =
E((S ; E(S)) 2 )= 2
, and the coe cient ofvariation is
2
D(S)
E(S)
= 1
p 2 n
This means that the most probable value for the length of the random vector at constant
increases as the square root of the number of variables and the relative width of variation
decreases with the reciprocal square root of parameters.