Evolution and Optimum Seeking

Random Strategies 97
maintained at least approximately during the course of the iterations. At the starting
point x (0) two random changes are made with step lengths s (0) and s (0) (1 + a), where
0 < a 1. If both samples are successful, for the next iteration s (1) = s (0) (1 + a) is
taken, i.e., the greater value. If only one sample yields an improvement in the objective
function, its step length is taken nally if no success is scored, s (1) remains equal to s (0) .
A reduction in s is only made if several consecutive trials are unsuccessful. This is
also the procedure of Maybach (1966). This adjustment to the local conditions assists the
strategy in achieving high convergence rates but reduces the chances of locating global
optima among several local ones. For this reason a sample with a signi cantly larger step
length (a > 1) should be included from time to time. Numerical tests show that the
computation cost, or number of trials, actually only increases linearly with the number
of variables. Schumer and Steiglitz have tested this using the model functions F3 and
F4(x) =
A comparison with a Newton-Raphson strategy, inwhich the partial rst and second
derivatives are determined numerically and the cost increases as O(n 2 ), favors the random
method when n>78 for F3 and when n>2 for F4. For the second, biquadratic model
function, Nelder and Mead (1965) state that the numberoftrialsorfunctionevaluations
in their simplex strategy grows as O(n 2:11 ), so that the sequential random method is
superior from n>10. White and Day (1971) report numerical tests in which the cost in
iterations with Schumer's strategy increases more sharply than linearly with n, whereas
a modi cation by White (1970) shows exact linear dependence. A comparison with the
strategy of Fletcher and Powell (1963) favors the latter, especially for truly quadratic
functions.
Rechenberg (1973), with an n-dimensional normal distribution (see Chap. 5, Sect. 5.1),
reaches almost the same theoretical results as Schumer for the circumferential distribution,
if one notes that the overall step length
tot =
vu
u
t nX
i=1
nX
i=1
x 4
i
2
i = p n
for equal variances 2
i = 2 in each random component zi is proportional to the square
root of the number of variables. The reason for this lies in the property of Euclidean
space that, as the number of dimensions increases, the volume of a hypersphere becomes
concentrated more and more in the boundary region near the surface. Rechenberg's adaptation
rule is founded on the relation between optimal variance and probability of success
derived from two essentially di erent models of the objective function. The adaptation
rule which is thereby formulated makes the frequency and size of the increments respectively
dependent on the number of variables and independent of the structure of the
objective function. This will be discussed in more detail in Chapter 5, Section 5.1.
Convergence proofs for the sequential random strategy have been given by Matyas
(1965, 1967) and Rechenberg (1973) only for the case of constant variance 2 . Gurin
(1966) has proved convergence also for stochastically perturbed objective functions. The