Darwin's Dangerous Idea - Evolution and the Meaning of Life

436 THE EMPEROR'S NEW MIND, AND OTHER FABLES
difficult to fool the expert judges—a problem, once more, of not having a
proper "sword-in-the-stone" feat to settle the issue. Holding a conversation or
winning a chess match is not a suitable feat, the former because it is too
open-ended for a contestant to secure unambiguous victory in spite of its
severe difficulty, and the latter because it is demonstrably within the power
of a machine after all. Might the implications of Godel's Theorem provide a
better contest? Suppose we put a mathematician in box A and a computer—
any computer you like—in box B, and ask each of them questions about the
truth and falsehood of sentences of arithmetic. Would this be a test that
would surely unmask the machine? The trouble is that human mathematicians
all make mistakes, and Godel's Theorem offers no verdict at all about the
likelihood, let alone impossibility, of less-than-perfect truth detection by an
algorithm. It does not appear, then, that there is any fair arithmetic test we
can put to the boxes that will clearly distinguish the man from the machine.
This difficulty had been widely seen as systematically blocking any argument
from Godel's Theorem to the impossibility of AI. Certainly everybody
in AI has always known about Godel's Theorem, and they have all continued,
unworried, with their labors. In fact, Hofstadter's classic Godel Escher Bach
(1979) can be read as the demonstration that Godel is an unwilling champion
of AI, providing essential insights about the paths to follow to strong AI, not
showing the futility of the field. But Roger Penrose, Rouse Ball Professor of
Mathematics at Oxford, and one of the world's leading mathematical
physicists, thinks otherwise. His challenge has to be taken seriously, even if,
as I and others in AI are convinced, he is making a fairly simple mistake. When
Penrose's book appeared, I pointed out the problem in a review: his argument
is highly convoluted, and bristling with details of physics and mathematics,
and it is unlikely that such an enterprise would succumb to a single,
crashing oversight on the part of its creator—that the argument could be
'refuted' by any simple observation. So I am reluctant to credit my observation
that Penrose seems to make a fairly elementary error right at the
beginning, and at any rate fails to notice or rebut what seems to be an
obvious objection. [Dennett 1989b.]
My surprise and disbelief were soon echoed, first by the usual assortment
of commentators to a target article (based on his book) by Penrose in Behavioral
and Brain Sciences, and then by Penrose in turn. In "The Nonalgorithmic
Mind" (1990), Penrose's reply to his critics, he expressed mild
astonishment at the strong language some of them used: "quite fallacious,"
"wrong," "lethal flaw" and "inexplicable mistake," "invalid," "deeply flawed."
The AI community was, not surprisingly, united in its dismissal of Penrose's
The Library of Toshiba 437
argument, but, in Penrose's eyes, they didn't agree on what "the" lethal flaw
was. This was itself a measure of how widely he had missed the mark, since
the critics had found many different ways of zeroing in on one big misunderstanding,
about the very nature of AI and its use of algorithms.
2. THE LIBRARY OF TOSHIBA
The people who are going to like the book best, however, will probably
be those who don't understand it. As an evolutionary biologist, I have
learned over the years that most people do not want to see themselves
as lumbering robots programmed to ensure the survival of their genes. I
don't think they will want to see themselves as digital computers either.
To be told by someone with impeccable scientific credentials that they
are nothing of the kind can only be pleasing.
—JOHN MAYNARD SMITH 1990 (
review of Penrose )
Consider the set of all Turing machines—in other words, the set of all
possible algorithms. Or, rather, to ease the task of imagination, consider
instead a Vast but finite subset of them, relativized to a particular language,
and consisting of "volumes" of a particular length: the set of all possible
strings of 0 and 1 (bit strings), up to the length of one megabyte (eight
million 0's and l's). Consider the reader of these strings to be my old laptop
computer, a Toshiba T-1200, with its twenty-megabyte hard disk (we'll
prohibit using any additional memory, just for finiteness' sake). It should
come as no surprise that the Vast majority of these bit strings do nothing at
all worth mentioning if an attempt is made to "run" them as programs on the
Toshiba. Programs, after all, are not random strings of bits, but highly
designed sequences of bits, the products of thousands of hours of R and D.
The fanciest program that ever could be is still something that can be
expressed as one or another string of 0's and l's, and although my old Toshiba
is too small to run some of the truly huge programs that have been devised, it
is quite capable of running a handsome and representative subset of them:
word-processors, spread sheets, chess-players, Artificial Life simulations,
logic-proof-checkers, and, yes, even a few automatic arithmetic-truthprovers.
Call any such runnable program, actual or envisaged, an interesting
program (it is roughly analogous to a readable book, actual or imaginary, in
the Library of Babel, or a viable genotype in the Library of Mendel). We
don't have to worry about the boundary separating the interesting from the
uninteresting; when in doubt, throw it out. No matter how we rule, there are
Vastly many interesting programs in the Library of