An Introduction to Genetic Algorithms - Boente

Chapter 3: Genetic Algorithms in Scientific Models
enough energy in its internal store. Agents reproduce by copying their genomes (subject to mutation at low
probability). In addition to this direct copying, two spatially nearby agents can together produce offspring via
crossover. There is no explicit given ("exogenous") fitness function for evaluating a genome, as there was in
Hinton and Nowlan's model and as there are in most engineering applications of GAs. Instead, the fitness of
an agent (as well as the rate at which a population turns over) is "endogenous" it emerges from many actions
and interactions over the course of the agent's lifetime. This feature distinguishes many GAs used in
artificial−life models from those used in engineering applications.
At each time step t in an agent's life, the agent uses its evaluation network to evaluate its current state. The
difference between the current evaluation and that computed at step t 1 serves as a reinforcement signal
judging the action the agent took at t 1, and is used to modify the weights in the action network. The hope is
that an agent will learn to act in ways that lead to "better" states, where "better" is defined by that particular
agent's inborn evaluation network. After this learning step, the agent uses its modified action network to
determine its next action.
Ackley and Littman observed many interesting phenomena in their experiments with this model. First, they
wanted to see whether or not the combination of evolution and learning produced any survival advantage to a
population. They measured the "performance" of the system by determining how long a population can
survive before becoming extinct, and they compared the performances of ERL (evolution plus learning), E
(evolution alone with no learning), L (learning alone with no evolution—i.e., no reproduction, mutation, or
crossover), and two controls: F (fixed random weights) and B ("Brownian" agents that ignore any inputs and
move at random). This kind of comparison is typical of the sort of experiment that can be done with a
computer model; such an experiment would typically be impossible to carry out with real living systems.
Figure 3.7: The distribution of population lifetimes for 100 runs for the ERL strategy and four variations:
evolution only (E), learning only (L), fixed random weights (F), and random (Brownian) movements (B).
Each plot gives the percentage of runs on which the population became extinct by a certain number of time
steps. For example, the point marked with a diamond indicates that 60% of the E (evolution only) populations
were extinct by ~ 1500 time steps. (Reprinted from Christopher G. Langton et al., eds., Artificial Life:
Volume II; © 1992 Addison−Wesley Publishing Company, Inc. Reprinted by permission of the publisher.)
The comparisons were made by doing a number of runs with each variation of the model, letting each run go
until either all agents had died out or a million time steps had taken place (at each time step, each agent in the
population moves), and recording, for each variation of the model, the percent of runs in which the population
had gone extinct at each time step. Figure 3.7 plots the results of these comparisons. The x axis gives a log
scale of time, and the y axis gives the percent of populations that had gone extinct by a given time.
Figure 3.7 reveals some unexpected phenomena. Evolution alone (E) was not much better than fixed random
initial weights, and, strangely, both performed considerably worse than random Brownian motion. Learning
seemed to be important for keeping agents alive, and learning alone (L) was almost as successful as evolution
and learning combined (ERL). However, ERL did seem to have a small advantage over the other strategies.
Ackley and Littman (1992, p. 497) explained these phenomena by speculating that "it is easier to generate a
73