Evolution and Optimum Seeking

82 Hill climbing Strategies
McCormick and Ritter (1972, 1974)
Murray (1972a,b)
Murtagh (1970)
Murtagh and Sargent (1970)
Oi, Sayama, and Takamatsu (1973)
Oren (1973)
Ortega and Rheinboldt (1972)
Pierson and Rajtora (1970)
Powell (1969, 1970a,b,c,g, 1971, 1972a,b,c,d)
Rauch (1973)
Ribiere (1970)
Sargent and Sebastian (1972, 1973)
Shanno and Kettler (1970a,b)
Spedicato (1973)
Tabak (1969)
Tokumaru, Adachi, and Goto (1970)
Werner (1974)
Wolfe (1967, 1969, 1971)
Many of the di erently sophisticated strategies, e.g., the classes or families of similar
methods de ned by Broyden (1970b,c) and Huang (1970), are theoretically equivalent.
They generate the same conjugate directions v (k) and, with an exact line search, the same
sequence x (k) of iteration points if F (x) is quadratic. Dixon (1972c) even proves this
identity for more general objective functions under the condition that no term of the
sequence H (k) is singular.
The important nding that under certain assumptions convergence can also be achieved
without line searches is attributed to Wolfe (1967). A recursion formula satisfying these
conditions is as follows:
H (k+1) = H (k) + B (k)
where
B (k) = (y(k) ; H (k) z (k) )(y (k) ; H (k) z (k) ) T
(y (k) ; H (k) z (k) ) z (k)T
(3.33)
The formula was proposed independently by Broyden (1967), Davidon (1968, 1969), Pearson
(1969), and Murtagh and Sargent (1970) (see Powell, 1970a). The correction matrix
B (k) has rank one, while A (k) in Equation (3.31) is of rank two. Rank one methods, also
called variance methods byDavidon, cannot guarantee that H (k) remains positive-de nite.
It can also happen, even in the quadratic case, that H (k) becomes singular or B (k) increases
without bound. Hence in order to make methods of this type useful in practice
anumber of additional precautions must be taken (Powell, 1970a Murray, 1972c). The
following compromise proposal
H (k+1) = H (k) + A (k) + (k) B (k)
(3.34)
where the scalar parameter (k) > 0 can be freely chosen, is intended to exploit the advantages
of both concepts while avoiding their disadvantages (e.g., Fletcher, 1970b). Broyden