Darwin's Dangerous Idea - Evolution and the Meaning of Life

210 BIOLOGY IS ENGINEERING
tigation of heuristic algorithms. But there is also a tradition within computer
science and mathematics of contrasting heuristic methods with algorithmic
methods: heuristic methods are risky, not guaranteed to yield results, whereas
algorithms come with a guarantee. How do we resolve this "contradiction"?
There is no contradiction at all. Heuristic algorithms are, like all algorithms,
mechanical procedures that are guaranteed to do what they do, but what they
do is engage in risky search! They are not guaranteed to find anything—or at
least they are not guaranteed to find the specific thing sought in the amount of
time available. But, like well-run tournaments of skill, good heuristic
algorithms tend to yield highly interesting, reliable results in reasonable
amounts of time. They are risky, but the good ones are good risks indeed.
You can bet your life on them (Dennett 1989b). Failure to appreciate the fact
that algorithms can be heuristic procedures has misled more than a few critics
of Artificial Intelligence, In particular, it has misled Roger Penrose, whose
views will be the topic of chapter 15.
Samuel saw that the Vast space of checkers could only be feasibly explored
by a process that riskily pruned the search tree, but how do you go
about constructing the pruning and choosing demons to do this job? What
readily programmable stop-looking-now rules or evaluation functions would
have a better-than-chance power to grow a search tree in wise directions?
Samuel was searching for a good algorithmic searching method. He proceeded
empirically, beginning by devising ways of mechanizing whatever
obvious rules of thumb he could think of. Look before you leap, of course,
and learn from your mistakes, so the system should have a memory in which
to store past experience. "Rote learning" carried the prototype quite far, by
simply storing thousands of positions it had already encountered and seen the
fruits of. But rote learning can only take you so far; Samuel's program
confronted rapidly diminishing returns when it had stored in the neighborhood
of a million words of description of past experience and began to be
overcome with indexing and retrieval problems. When higher or more
versatile performance is required, a different strategy of design has to kick in:
generalization.
Instead of trying to find the search procedure himself, Samuel tried to get
the computer to find it. He wanted the computer to design its own evaluation
function, a mathematical formula—a polynomial—that would yield a
number, positive or negative, for every move it considered, such that, in
general, the higher the number, the better the move. The polynomial was to
be concocted of lots of pieces, each contributing positively or negatively,
multiplied by one coefficient or another, and adjusted to various other
circumstances, but Samuel had no idea what sort of concoction would work
well. He made some thirty-eight different chunks—"terms"—and threw them
into a "pool." Some of the terms were intuitively valuable, such as those
giving points for increased mobility or potential captures, but others were
more or less off the wall—such as "DYKE: the parameter is credited with
The Computer That Learned to Play Checkers 211
1 for each string of passive pieces that occupy three adjacent diagonal
squares." At any one time, sixteen of the terms were thrown together into the
working genome of the active polynomial and the rest were idle. By a lot of
inspired guesswork and even more inspired tuning and tinkering, Samuel
devised rules for elimination from the tournament, and found ways of
keeping the brew stirred up, so that the trial-and-error process was likely to
hit upon good combinations of terms and coefficients and recognize them
when it did. The program was divided into Alpha, a rapidly mutating pioneer,
and Beta, a conservative opponent that played the version that had won the
most recent game. "Alpha generalizes on its experience after each move by
adjusting the coefficients in its evaluation polynomial and by replacing terms
which appear to be unimportant by new parameters drawn from the reserve
list" (Samuel 1964, p. 83).
At the start an arbitrary selection of 16 terms was chosen and all terms
were assigned equal weights__ During [the early rounds] a total of 29
different terms was discarded and replaced, the majority of these on two
different occasions __ The quality of the play was extremely poor. During
the next seven games there were at least eight changes made in the top
listing involving five different terms __Quality of play improved steadily
but the machine still played rather badly ___ Some fairly good amateur
players who played the machine during this period [after seven more
games] agreed that it was 'tricky but beatable'. [Samuel 1964, p. 89]
Samuel noted (p. 89) that, although the learning at this early stage was
surprisingly fast, it was "quite erratic and none too stable." He was discovering
that the problem space being explored was a rugged fitness landscape in
which a program using simple hill-climbing techniques tended to fall into
traps, instabilities, and obsessive loops from which the program could not
recover without a helping nudge or two from its designer. He was able to
recognize the "defects" in his system responsible for these instabilities and
patch them. The final system—the one that beat Nealey—was a Rube Goldberg
amalgam of rote learning, kludges, 15 and products of self-design that
were quite inscrutable to Samuel himself.
15. Pronounced to rhyme with "stooge," a kludge is an ad hoc or jury-rigged patch or
software repair. Purists spell this slang word "kluge," drawing attention to its (likely)
etymology in the deliberate mispronunciation of the German word klug(e), meaning
"clever"; but according to The New Hacker's Dictionary (Raymond 1993), the term may
have an earlier ancestor, deriving from the Kluge paper feeder, an "adjunct to mechanical
printing presses" in use as early as 1935. In its earlier use, it named "a complex and
puzzling artifact with a trivial function." The mixture of esteem and contempt hackers
exhibit for kluges ("How couid anything so dumb be so smart!") perfectly reproduces
the attitude of biologists when they marvel at the perversely intricate solutions Mother
Nature so often discovers.