Evolution and Optimum Seeking

Random Strategies 89
parts is taken as an approximation to the global optimum. However, for n-dimensional
interpolations the cost increases rapidly with n thisscheme thus looks impractical for
more than two variables. To work with several, randomly chosen starting points and to
compare each of the local minima (or maxima) obtained is usually regarded as the only
course of action for determining the global optimum with at least a certain probability
(so-called multistart techniques). Proposals along these lines have beenmadeby, among
others, Gelfand and Tsetlin (1961), Bromberg (1962), Bocharov andFeldbaum (1962),
Zellnik, Sondak, and Davis (1962), Krasovskii (1962), Gurin and Lobac (1963), Flood and
Leon (1964, 1966), Kwakernaak (1965), Casey and Rustay (1966), Weisman and Wood
(1966), Pugh (1966), McGhee (1967), Crippen and Scheraga (1971), and Brent (1973).
A further problem faces deterministic strategies if the calculated or measured values
of the objective function are subject to stochastic perturbations. In the experimental
eld, for example in the on-line optimum search, or for control of the optimal conditions
in processes, perturbations must be taken into account from the start (e.g., Tovstucha,
1960 Feldbaum, 1960, 1962 Krasovskii, 1963 Medvedev, 1963, 1968 Kwakernaak, 1966
Zypkin, 1967). However, in computational optimization too, where the objective function
is analytically speci ed, a similar e ect arises because of rounding errors (Brent, 1973),
especially if one uses hybrid analogue computers for solving functional optimization problems
(e.g., Gilbert, 1967 Korn and Korn, 1964 Bekey and Karplus, 1971). A simple, if
expensive (in the sense of cost in computations or trials) method of dealing with this is the
repetition of measurements until a de nite conclusion is possible. This is the procedure
adopted by Box and Wilson (1951) in the experimental gradient method, and by Box
(1957) in his EVOP strategy. Instead of a xed number of repetitions, which while on
the safe side may be unnecessarily high, one can follow the concept of sequential analysis
of statistical data (Wald, 1966 see also Zigangirov, 1965 Schumer, 1969 Kivelidi and
Khurgin, 1970 Langguth, 1972), which istomake only as many trials as the trial results
seem to make absolutely necessary. More detailed investigations on this subject havebeen
made, for example, by Mlynski (1964a,b, 1966a,b).
As opposed to attempting to improve the decisive data, Brooks and Mickey (1961)
have found that one should work with the minimum number of n + 1 comparison points
in order to determine a gradient direction, even if this is a perturbed one. One must
however depart from the requirement thateach step should yield a success, or even the
locally greatest success. The motto that following locally the best possible route seldom
leads to the best overall result is true not only for rst order gradient strategies but also for
Newton and quasi-Newton methods . Harkins (1964), for example, maintains that inexact
line searches not only do not worsen the convergence of a minimization procedure but in
some cases actually improve it. Similar experiences led Davies, Swann, and Campey in
their strategy (see Chap. 3, Sect. 3.2.1.4) to make only one quadratic interpolation in
each direction. Also Spendley, Hext, and Himsworth (1962), in the formulation of their
simplex method, which generates only near-optimal directions, work on the assumption
that random decisions are not necessarily a total disadvantage (see also Himsworth, 1962).
Based on similar arguments, the modi cation of this strategy by M.J.Box (1965) sets
up the initial simplex or complex by means of random numbers. Imamura et al. (1970)
even go so far as to superimpose arti cial stochastic variations on an objective function