Evolution and Optimum Seeking

Chapter 6
Comparison of Direct Search
Strategies for Parameter
Optimization
6.1 Di culties
The vast and steadily increasing number of optimization methods necessarily raises the
question of which is the best strategy. There seems to be no unique answer. If indeed
there were an optimal optimization method all the others would be super uous and would
have been long ago forgotten.
Because of the strong competition between already existing strategies it is necessary
nowadays that whenever any proposal for a new method or variant is made, its advantages
and improvements compared to older strategies be displayed. The usual way is to refer to
a minimum problem for which the known methods fail to nd a solution whereas the new
proposal is successful. Or it is shown with reference to chosen examples that computation
time or iterations can be saved by using the new version. The series of publications
along these lines can in principle be continued inde nitely. With su cient insight into
the working of any strategy a special optimization problem can always be constructed for
which the strategy fails. Likewise for any problem a special method of solution can be
devised that is superior to the other procedures. One simply needs to exploit to the full
what one knows of the problem structure as contained in its mathematical formulation.
Progress in the eld of optimization methods does not, however, consist in developing
an individual method of solution for each problem or type of problem. A practitioner
would much rather manage with just one strategy, which can solve all the practically
occurring problems for as small a total cost as possible. But as yet there is no such
universal optimization method, and some authors doubt if there ever will be (Arrow and
Hurwicz, 1957). All the methods presently known can only be used without restriction
in particular areas of application. According to the nature of the particular problem,
one or another strategy o ers a more successful solution. The question of which isthe
best strategy is itself a kind of optimization problem. To be able to answer it objectively
an objective function would have tobeformulated for deciding which oftwo methods
165