Evolution and Optimum Seeking

Multidimensional Strategies 57
Anumerical strategy comparison by M.J. Box (1966) shows the method to be a very
e ective optimization procedure, in general superior both to the Hooke and Jeeves and
the Rosenbrock methods. However, the tests only refer to smooth objective functions with
few variables. If the number of parameters is large, the costly orthogonalization process
makes its inconvenient presence felt also in the DSC strategy.
Several suggestions have been made to date as to how to simplify the Gram-Schmidt
procedure and to reduce its susceptibilitytonumerical rounding error (Rice, 1966 Powell,
1968a Palmer, 1969 Golub and Saunders, 1970, Householder method).
Palmer replaces the conditions of Equation (3.21) by:
v (k+1)
i
=
8
><
>:
v (k)
i if
nP
j=1
nP
j=1
dj v (k)
s
nP
j = d
j=1
2
j for i =1
nP
di;1
j=i
dj v (k)
j
; v (k)
i;1
nP
j=i
d 2
j
!
s
nP
=
j=i
d 2
j
nP
j=i;1
d 2
j for i = 2(1)n
d 2
j = 0 otherwise
He shows that even if no success was obtained in one of the directions v (k)
i , that is di =0,
the new vectors v (k+1)
i
v (k+1)
i+1
for all i = 1(1)n still span the complete parameter space, because
is set equal to ;v (k)
i .Thus the algorithm does not need to be restricted to directions
for which di >", as happens in the algorithm with Gram-Schmidt orthogonalization.
The signi cant advantage of the revised procedure lies in the fact that the number
of computational operations remains only of the order O(n 2 ). The storage requirement
is also somewhat less since one n n matrix as an intermediate storage area is omitted.
For problems with linear constraints (equalities and inequalities) Box, Davies, and Swann
(1969) recommend a modi cation of the orthogonalization procedure that works in a
similar way to the method of projected gradients of Rosen (1960, 1961) (see also Davies,
1968). Non-linear constraints (inequalities) can be handled with the created response
surface technique devised by Carroll (1961), which is one of the penalty function methods.
Further publications on the DSC strategy, also with comparison tests, are those of
Swann (1969), Davies and Swann (1969), Davies (1970), and Swann (1972). Hoshino
(1971) observes that in a narrow valley the search causes zigzag movements. His remedy
for this is to add a further search, again in direction v (k)
1 , after each setofn line searches.
With the help of two examples, for n =2andn =3,heshows the accelerating e ect of
this measure.
3.2.1.5 Simplex Strategy of Nelder and Mead
There are a group of methods called simplex strategies that work quite di erently to
the direct search methods described so far. In spite of their common name they have
nothing to do with the simplex method of linear programming of Dantzig (1966). The
idea (Spendley, Hext, and Himsworth, 1962) originates in an attempt to reduce, as much
as possible, the number of simultaneous trials in the experimental identi cation procedure