Evolution and Optimum Seeking

54 Hill climbing Strategies
Rosenbrock's strategy can be found in Storey (1962), and Storey and Rosenbrock (1964).
Among them is also a discretized functional optimization problem. For unconstrained
problems there exists the code of Machura and Mulawa (1973). The Gram-Schmidt
orthogonalization has been programmed, for example, by Clayton (1971).
Lange-Nielsen and Lance (1972) have proposed, on the basis of numerical experiments,
two improvements in the Rosenbrock strategy. The rst involves not setting constant
step lengths at the beginning of a cycle or after each orthogonalization, but rather
modifying them and simultaneously scaling them according to the successes and failures
during the preceding cycle. The second improvement concerns the termination criterion.
Rosenbrock's original version is replaced by the simpler condition that, according to the
achievable computational accuracy, several consecutive trials yield the same value of the
objective function.
3.2.1.4 Strategy of Davies, Swann, and Campey (DSC)
A combination of the Rosenbrock idea of rotating coordinates with one dimensional search
methods is due to Swann (1964). It has become known under the name Davies-Swann-
Campey (abbreviated DSC) strategy. The description of the procedure given by Box,
Davies, and Swann (1969) di ers somewhat from that in Swann, and so several versions
of the strategy have arisen in the subsequent literature. Preference is given here to the
original concept of Swann, which exhibits some features in common with the method of
conjugate directions of Smith (1962) (see also Sect. 3.2.2). Starting from x (00) , a line
search is made in each of the unit directions v (0)
i
= ei for all i =1(1)n. This process is
followed by a one dimensional minimization in the direction of the overall success so far
achieved
v (0)
n+1 = x(0n) ; x (00)
kx (0n) ; x (00) k
with the result x (0n +1) .
The orthogonalization follows this, e.g., by the Gram-Schmidt method. If one of the
line searches was unsuccessful the new set of directions would no longer span the complete
parameter space. Therefore only those old direction vectors along which a prescribed
minimum distance has been moved are included in the orthogonalization process. The
other directions remain unchanged. The DSC method, however, places a further hurdle
before the coordinate rotation. If the distance covered in one iteration is smaller than
the step length used in the line search, the latter is reduced by a factor 10, and the next
iteration is carried out with the old set of directions.
After an orthogonalization, one of the new directions (the rst) coincides with that of
the (n+1)-th line search of the previous step. This can therefore also be interpreted as the
rst minimization in the new coordinate system. Only n more one dimensional searches
need be made to nish the iteration. As a termination criterion the DSC strategy uses
the length of the total vector between the starting point andendpoint of an iteration.
The search is ended when it is less than a prescribed accuracy bound.