Evolution and Optimum Seeking

Multidimensional Strategies 77
3.2.3.1 DFP: Davidon-Fletcher-Powell Method
(Quasi-Newton Strategy, Variable Metric Strategy)
Much greater interest has been shown for a group of second order gradient methods that
attempt to approximate the Hessian matrix and its inverse during the iterations only from
rst order data. This now extensive class of quasi-Newton strategies has grown out of
the work of Davidon (1959). Fletcher and Powell (1963) improved and translated it into
a practical procedure. The Davidon-Fletcher-Powell or DFP method and some variants
of it are also known as variable metric strategies. They are sometimes also regarded
as conjugate gradient methods, because in the quadratic case they generate conjugate
directions. For higher order objective functions this is no longer so. Whereas the variable
metric concept is to approximate Newton directions, this is not the case for conjugate
gradient methods. The basic recursion formula for the DFP method is
with
and
x (k+1) = x (k) + s (k) v (k)
v (k) = ;H (k)T rF (x (k) )
H (0) = I
H (k+1) = H (k) + A (k)
The correction A (k) to the approximation for the inverse Hessian matrix, H (k) ,is
derived from information collected during the last iteration thus from the change in the
variable vector
y (k) = x (k+1) ; x (k) = s (k) v (k)
and the change in the gradient vector
it is given by
z (k) = rF (x (k+1) ) ;rF (x (k) )
A (k) = y(k) y (k)T
y (k)T z (k) ; H (k) z (k) (H (k) z (k) ) T
z (k)T H (k) z (k)
(3.31)
The step length s (k) is obtained by a line search along v (k) (Equation (3.30)). Since
the rst partial derivatives are needed in any case they can be made use of in the one
dimensional minimization. Fletcher and Powell do so in the context of a cubic Hermitian
interpolation (see Sect. 3.1.2.3.4). A corresponding ALGOL program has been
published by Wells (1965) (for corrections see Fletcher, 1966 Hamilton and Boothroyd,
1969 House, 1971). The rst derivatives must be speci ed as functions, which is usually
inconvenient and often impossible. The convergence properties of the DFP method have
been thoroughly investigated, e.g., by Broyden (1970b,c), Adachi (1971), Polak (1971),
and Powell (1971, 1972a,b,c). Numerous suggestions have thereby been made for improvements.
Convergence is achieved if F (x) isconvex. Under stricter conditions it can
be proved that the convergence rate is greater than linear and the sequence of iterations