Evolution and Optimum Seeking

Chapter 4
Random Strategies
One group of optimization methods has been completely ignored in Chapter 3: methods in
which the parameters are varied according to probabilistic instead of deterministic rules
even the methods of stochastic approximation are deterministic. As indicated by the title
there is not one random strategy but many, someofwhich di er considerably from each
other.
It is common to resort to random decisions in optimization whenever deterministic
rules do not have the desired success, or lead to a dead end on the other hand random
strategies are often supposed to be essentially more costly. The opinion is widely held that
with careful thought leading to cleverly constructed deterministic rules, better results can
always be achieved than with decisions that are in some way made randomly. The strategies
that follow should show that randomness is not, however, the same as arbitrariness,
but can also be made to obey very re ned rules. Sometimes only this kind of method
solves a problem e ectively.
Profound considerations do not underlie all the procedures used in hill climbing strategies.
The cyclic choice of coordinate directions in the Gauss-Seidel strategy could just as
well be replaced by a random sequence. One can also consider increasing the number of
directions used. Since there is no good reason for preferring to search for the optimum
along directions parallel to the axes, one could also use, instead of only n di erent unit
vectors, any numberofrandomlychosen direction vectors. In fact, suggestions along
these lines have been made (Brooks, 1958) in order to avoid a premature termination of
the minimum search innarrow oblique valleys (compare Chap. 3, Sect. 3.2.1.1). Similar
concepts have been developed for example by O'Hagan and Moler (after Wilde and
Beightler, 1967), Emery and O'Hagan (1966), Lawrence and Steiglitz (1972), and Beltrami
and Indusi (1972), to improve the pattern search ofHookeandJeeves (1961, see
Chap. 3, Sect. 3.2.1.2). The limitation to a nite number of search directions is not only
a disadvantage in narrow oblique valleys but also at the border of the feasible region
as determined by inequality constraints. All the deterministic remedies against prematurely
ending the iteration sequence assume that more information can be gathered, for
example in the form of partial derivatives of the constraint functions (see Klingman and
Himmelblau, 1964 Glass and Cooper, 1965 Paviani and Himmelblau, 1969). Providing
this information usually means a high extra cost and is sometimes not possible at all.
87