An Introduction to Genetic Algorithms - Boente

competition in the d *···* partition in Grefenstette and Baker's example will not follow a two−armed−bandit
strategy.
The general idea here is that, roughly, the multi−armed−bandit strategy for competition among schemas
proceeds from low−order partitions at early times to higher−order partitions at later times. The extent to which
this describes the workings of actual GAs is not clear. Grefenstette (1991b) gives some further caveats
concerning this description of GA behavior. In particular, he points out that, at each new generation, selection
most likely produces a new set of biases in the way schemas are sampled (as in the fitness function given by
equation 4.5 above). Since the Schema Theorem gives the dynamics of schema sampling over only one time
step, it is difficult (if not impossible), without making unrealistic assumptions, to make any long−range
prediction about how the allocation of trials to different schemas will proceed. Because of the biases
introduced by selection, the static average fitnesses of schemas will not necessarily be correlated in any useful
way with their observed average fitnesses.
Implications for GA Performance
Chapter 4: Theoretical Foundations of Genetic Algorithms
In view of the solution to the Two−Armed Bandit problem, the exponential allocation of trials is clearly an
appropriate strategy for problems in which "on−line" performance is important, such as real−time control
problems. Less clear is its appropriateness for problems with other performance criteria, such as "Is the
optimal individual reliably discovered in a reasonable time?" There have been many cases in the GA literature
in which such criteria have been applied, sometimes leading to the conclusion that GAs don't work. This
demonstrates the importance of understanding what a particular algorithm is good at doing before applying it
to particular problems. This point was made eloquently in the context of GAs by De Jong (1993).
What is maximizing on−line performance good for? Holland's view is clear: the rough maximization of
on−line performance is what goes on in adaptive systems of all kinds, and in some sense this is how
"adaptation" is defined. In the realm of technology, on−line performance is important in control problems
(e.g., automatically controlling machinery) and in learning problems (e.g., learning to navigate in an
environment) in which each system action can lead to a gain or loss. It is also important in prediction tasks
(e.g., predicting financial markets) in which gains or losses are had with each prediction. Most of the GA
applications I discussed in chapter 2 had a slightly different performance criterion: to find a "reasonably good"
(if not perfect) solution in a reasonable amount of time. In other words, the goal is to satisfice (find a solution
that is good enough for one's purposes) rather than to optimize (find the best possible solution). This goal is
related to maximizing on−line performance, since on−line performance will be maximized if high−fitness
individuals are likely to be chosen at each step, including the last. The two−armed−bandit allocation strategy
maximizes both cumulative payoff and the amount of information the gambler gets for her money as to which
is the better arm. In the context of schemas, the exponential allocation strategy could be said to maximize, for
a given amount of time (samples), the amount of information gained about which part of the search space is
likely to provide good solutions if sampled in the future. Gaining such information is crucial to successful
satisficing. For true optimization, hybrid methods such as a GA augmented by a hill climber or other kinds of
gradient search have often been found to perform better than a GA alone (see, e.g., Hart and Belew 1996).
GAs seem to be good at quickly finding promising regions of the search space, and hill climbers are often
good at zeroing in on optimal individuals in those regions.
Although the foregoing schema analysis has suggested the types of problems for which a genetic algorithm
might be useful, it does not answer more detailed questions, such as "Given a certain version of the GA, what
is the expected time to find a particular fitness level?" Some approaches to such questions will be described
below.
92