Evolution and Optimum Seeking

Multidimensional Strategies 83
(1970b,c), Shanno (1970a,b), and Shanno and Kettler (1970) give criteria for choosing suitable
(k) .However, the mixed correction, also known as BFS or Broyden-Fletcher-Shanno
formula, cannot guarantee quadratic convergence either unless line searches are carried
out. It can be proved that there will merely be a monotonic decrease in the eigenvalues of
the matrix H (k) .From numerical tests, however, it turns out that the increased number
of iterations is usually more than compensated for by thesaving in function calls made
by dropping the one dimensional optimizations (Fletcher, 1970a). Fielding (1970) has designed
an ALGOL program following Broyden's work with line searches (Broyden, 1965).
With regard to the number of function calls it is usually inferior to the DFP method but
it sometimes also converges where the variable metric method fails. Dixon (1973) de nes
a correction to the chosen directions,
where
and
v (k) = ;H (k) rF (x (k) )+w (k)
w (0) =0
w (k+1) = w (k) + (x(k+1) ; x (k) ) T rF (x (k+1) )
(x (k+1) ; x (k) ) T z (k)
(x (k+1) ; x (k) )
by which, together with a matrix correction as given by Equation (3.35), quadratic convergence
can be achieved without line searches. He shows that at most n +2 function
calls and gradient calculations are required each time if, after arriving at v (k) = 0, an
iteration
x (k+1) = x (k) ; H (k) rF (x (k) )
is included. Nearly all the procedures de ned assume that at least the rst partial derivatives
are speci ed as functions of the variables and are therefore exact to the signi cant
gure accuracy of the computer used. The more costly matrix computations should
wherever possible be executed with double precision in order to keep down the e ect of
rounding errors.
Just two more suggestions for derivative-free quasi-Newton methods will be mentioned
here: those of Greenstadt (1972) and of Cullum (1972). While Cullum's algorithm, like
Stewart's, approximates the gradient vector by function value di erences, Greenstadt attempts
to get away from this. Analogously to Davidon's idea of approximating the Hessian
matrix during the course of the iterations from knowledge of the gradients, Greenstadt
proposes approximating the gradients by using information from objective function values
over several subiterations. Only at the starting point must a di erence scheme for the rst
partial derivatives be applied. Another interesting variable metric technique described by
Elliott and Sworder (1969a,b, 1970) combines the concept of the stochastic approximation
for the sequence of step lengths with the direction algorithms of the quasi-Newton
strategy.
Quasi-Newton strategies of degree one are especially suitable if the objective function
is a sum of squares (Bard, 1970). Problems of minimizing a sum of squares arise
for example from the problem of solving systems of simultaneous, non-linear equations,