An Introduction to Genetic Algorithms - Boente

Chapter 2: Genetic Algorithms in Problem Solving
space the GA searched was thus 2 128 —far too large for any kind of exhaustive search.
In our main set of experiments, we set N = 149 (chosen to be reasonably large but not computationally
intractable). The GA began with a population of 100 randomly generated chromosomes (generated with some
initial biases—see Mitchell, Crutchfield, and Hraber 1994a, for details). The fitness of a rule in the population
was calculated by (i) randomly choosing 100 ICs (initial configurations) that are uniformly distributed over Á
Î [0.0,1.0], with exactly half with ÁÁc and half with ÁÁc, (ii) running the rule on each IC either until it arrives
at a fixed point or for a maximum of approximately 2N time steps, and (iii) determining whether the final
pattern is correct—i.e., N zeros for Á0Ác and N ones for Á0Ác. The initial density, Á0, was never exactly since
N was chosen to be odd. The rule's fitness, f100, was the fraction of the 100 ICs on which the rule produced the
correct final pattern. No partial credit was given for partially correct final configurations.
A few comments about the fitness function are in order. First, as was the case in Hillis's sorting−networks
project, the number of possible input cases (2 149 for N = 149) was far too large to test exhaustively. Instead,
the GA sampled a different set of 100 ICs at each generation. In addition, the ICs were not sampled from an
unbiased distribution (i.e., equal probability of a one or a zero at each site in the IC), but rather from a flat
distribution across Á Î [0,1] (i.e., ICs of each density from Á = 0 to Á = 1 were approximately equally
represented). This flat distribution was used because the unbiased distribution is binomially distributed and
thus very strongly peaked at . The ICs selected from such a distribution will likely all have , the
hardest cases to classify. Using an unbiased sample made it too difficult for the GA to ever find any
high−fitness CAs. (As will be discussed below, this biased distribution turns out to impede the GA in later
generations: as increasingly fit rules are evolved, the IC sample becomes less and less challenging for the
GA.)
Our version of the GA worked as follows. In each generation, (i) a new set of 100 ICs was generated, (ii) f100
was calculated for each rule in the population, (iii) the population was ranked in order of fitness, (iv) the 20
highest−fitness ("elite") rules were copied to the next generation without modification, and (v) the remaining
80 rules for the next generation were formed by single−point crossovers between randomly chosen pairs of
elite rules. The parent rules were chosen from the elite with replacement—that is, an elite rule was permitted
to be chosen any number of times. The offspring from each crossover were each mutated twice. This process
was repeated for 100 generations for a single run of the GA. (More details of the implementation are given in
Mitchell, Crutchfield, and Hraber 1994a.)
Note that this version of the GA differs from the simple GA in several ways. First, rather than selecting
parents with probability proportional to fitness, the rules are ranked and selection is done at random from the
top 20% of the population. Moreover, all of the top 20% are copied without modification to the next
generation, and only the bottom 80% are replaced. This is similar to the selection method—called "(¼ +
»)"—used in some evolution strategies; see Back, Hoffmeister, and Schwefel 1991.
This version of the GA was the one used by Packard (1988), so we used it in our experiments attempting to
replicate his work (Mitchell, Hraber, and Crutchfield 1993) and in our subsequent experiments. Selecting
parents by rank rather than by absolute fitness prevents initially stronger individuals from quickly dominating
the population and driving the genetic diversity down too early. Also, since testing a rule on 100 ICs provides
only an approximate gauge of the true fitness, saving the top 20% of the rules was a good way of making a
"first cut" and allowing rules that survive to be tested over more ICs. Since a new set of ICs was produced
every generation, rules that were copied without modification were always retested on this new set. If a rule
performed well and thus survived over a large number of generations, then it was likely to be a genuinely
better rule than those that were not selected, since it was tested with a large set of ICs. An alternative method
would be to test every rule in each generation on a much larger set of ICs, but this would waste computation
time. Too much effort, for example, would go into testing very weak rules, which can safely be weeded out
early using our method. As in most applications, evaluating the fitness function (here, iterating each CA) takes
37