Evolution and Optimum Seeking

Numerical Comparison of Strategies 229
accuracy. Bykeeping the starting conditions the same but rotating the coordinates with
respect to the contours of the objective function (Problem 3.6), or slightly tilting the
contours with respect to the coordinate axes (Problem 3.5), or both together (Problem
3.4), one could easily cause all the line searches to fail. On the other hand the strategies
without line searches would not be impaired by these changes. Thus the advantage of
selected directions can turn into a disadvantage. These coordinated strategies can never
solve the problem referred to, whereas, as we have seen, the strategies that have a large
set of search directions at their disposal only fail when a particular number of variables
is exceeded. Problems 3.4 and 3.6 are therefore suitable for assessing the reliability of
simplex, complex, and evolution strategies, but not for the other methods. Together they
belong to the type of problems which Himmelblau designates as \pathological."
Leaning more to the conservative side are the several times continuously di erentiable
objective functions of Problems 3.1, 3.2, 3.3, and 3.7. The rst two problems were tackled
successfully by all the strategies for any number of variables. The simplex method did,
however, need at least one restart for Problem 3.1 with n 100. For 135 variables it
exceeded the time limit before Problems 3.1 and 3.2 were solved to su cient accuracy.
Problem 3.3 gave trouble to several search procedures when there were 10 or more
variables. The coordinate strategies were the rst to fail. For only n = 10, the step
lengths of the line searches would have had to be smaller than allowed by thenumber
precision of the computer used. At n = 30, the DSC strategy with Gram-Schmidt
orthogonalization also ends without having located the minimum accurately enough. The
simplex method with one restart still found the solution for n = 30, but the complex
strategy failed here, either by premature termination of the search orbyreaching the
maximum permitted computation time. Problem 3.3, because the cost per objective
function evaluation increases as O(n 2 ), requires the longest computation times for its
solution. Since the objective function also took O(n 2 ) units of storage, this problem could
not be used for more than 30 variables.
Problem 3.7, like the analogous Problem 2.31, gave trouble to the two quadratically
convergent strategies. The method of Powell was only successful for n = 3. For more
variables it became stuck in the search process without the termination rule taking e ect.
The variable metric strategy behaved in just the same way. For n 30, it no longer
came as near as required to the optimum. Under the stricter conditions of the second set
of tests it failed already at n = 5. With both methods fatal execution errors occurred
during the search. No other direct search strategies had any di culty with Problem 3.7,
which is a simple 10th order polynomial. Only the simplex method would not have found
the solution su ciently accurately without the restart rule. For n = 100, it reached the
time limit before the search simplex had collapsed for the rst time.
The advantage shown by the complex strategy was due to the complex's having 2 n
vertices, which is almost twice as many asthen + 1 of the simplex. An attempt to solve
Problems 3.1 to 3.10 for n = 30 with a complex constructed of 40 points failed completely.
The search ended, in every case, without having reached the required accuracy.
How do the computation times compare when the problems are no longer only quadratically
non-linear? For solving the \pathological" Problems 3.4 to 3.6 all the methods with
a line search take about the same times, with the same number of variables, as they do