Evolution and Optimum Seeking

Chapter 3
Hill climbing Strategies
In this chapter some of the direct, mathematical parameter optimization methods will
be treated in more detail for static, non-discrete, non-stochastic, mostly unconstrained
functions. They come under the general heading of hill climbing strategies because their
manner of searching for a maximum corresponds closely to the intuitive way a sightless
climber might feel his way from a valley up to the highest peak of a mountain. For
minimum problems the sense of the displacements is simply reversed, otherwise uphill or
ascent and downhill or descent methods (Bach, 1969) are identical. Whereas methods of
mathematical programming are dominant in operations research and the special methods
of functional optimization in control theory, the hill climbing strategies are most frequently
applied in engineering design. Analytic methods often prove unsuitable in this eld
Because the assumptions are not satis ed under which necessary conditions for
extrema can be stated (e.g., continuity of the objective function and its derivatives)
Because there are di culties in carrying out the necessary di erentiations
Because a solution of the equations describing the conditions does not always lead
to the desired optimum (it can be a local minimum, maximum, or saddle point)
Because the equations describing the conditions, in general a system of simultaneous
non-linear equations, are not immediately soluble
To what extent hill climbing strategies take care of these particular characteristics
depends on the individual method. Very thorough presentations covering some topics can
be found in Wilde (1964), Rosenbrock and Storey (1966), Wilde and Beightler (1967),
Kowalik and Osborne (1968), Box, Davies, and Swann (1969), Pierre (1969), Pun (1969),
Converse (1970), Cooper and Steinberg (1970), Ho mann and Hofmann (1970), Beveridge
and Schechter (1970), Aoki (1971), Zahradnik (1971), Fox (1971), Cea (1971), Daniel
(1971), Himmelblau (1972b), Dixon (1972a), Jacoby, Kowalik and Pizzo (1972), Stark and
Nicholls (1972), Brent (1973), Gottfried and Weisman (1973), Vanderplaats (1984), and
Papageorgiou (1991). More variations or theoretical and numerical studies of older methods
can be found as individual publications in a wide variety of journals, or in the volumes
of collected articles such asGraves and Wolfe (1963), Blakemore and Davis (1964), Lavi
23