Darwin's Dangerous Idea - Evolution and the Meaning of Life

442 THE EMPEROR'S NEW MIND, AND OTHER FABLES The Library of Toshiba 443
That is interesting, but it doesn't help us much. Lots of interesting things
can be proved, mathematically, about each and every member of various sets
of algorithms. Applying that knowledge in the real world is another matter,
and that is the blind spot that led Penrose to overlook AI altogether, instead
of refuting it, as he hoped. This has come out quite clearly in his subsequent
attempts at reformulation of his claim in response to his critics.
Given any particular algorithm, that algorithm cannot be the procedure
whereby human mathematicians ascertain mathematical truth. Hence
humans are not using algorithms at all to ascertain truth. [Penrose 1990,
p. 696.]
Human mathematicians are not using a knowably sound algorithm in order
to ascertain mathematical truth. [Penrose 1991.]
In the more recent of these, he goes on to consider and close various
"loopholes," of which two in particular concern us. mathematicians might be
using "a horrendously complicated unknowable algorithm X" or "an unsound
(but presumably approximately sound) algorithm Y." Penrose presents these
loopholes as if they were ad hoc responses to the challenge of Godel's
Theorem, instead of the standard working assumptions of AI. Of the first he
says:
This seems to be totally at variance with what mathematicians seem actually
to be doing when they express their arguments in terms that can ( at
least in principle ) be broken down into assertions that are 'obvious', and
agreed by all. I would regard it as far-fetched in the extreme to believe that
it is really the horrendous unknowable X, rather than these simple and
obvious ingredients [emphasis added], that lies lurking behind all our
mathematical understanding. [Penrose 1991]
These "ingredients" are indeed wielded by us all in an apparently nonalgorithmic
way, but this phenomenological fact is misleading. Penrose pays
careful attention to what it is like to be a mathematician, but he overlooks a
possibility—indeed, a likelihood—that is familiar to AI researchers: the
possibility that underlying our general capacity to deal with such "ingredients"
is a heuristic program of mind-boggling complexity. Such a complicated
algorithm would approximate the competence of the perfect
understander, and be "invisible" to its beneficiary. Whenever we say we
solved some problem "by intuition," all that really means is we don't know
how we solved it. The simplest way of modeling "intuition" in a computer is
simply denying the computer program any access to its own inner workings.
Whenever it solves a problem, and you ask it how it solved the problem, it
should respond: "I don't know; it just came to me by intuition" (Dennett
1968).
He goes on to dismiss his second loophole (the unsound algorithm) by
claiming (1991): "Mathematicians require a degree of rigour that makes such
heuristic arguments unacceptable—so no such known procedure of this kind
can be the way that mathematicians actually operate." This is a more
interesting mistake, for with it he raises the prospect that the crucial
empirical test would be not to put a single mathematician "in the box" but the
whole mathematical community! Penrose sees the theoretical importance of
the added power that human mathematicians obtain by pooling their
resources, communicating with each other, and hence becoming a sort of
single giant mind that is hugely more reliable than any one homunculus we
might put in the box. It is not that mathematicians have fancier brains than
the rest of us (or than chimpanzees) but that they have mind-tools—the
social institutions in which mathematicians present each other their proofs,
check each other out, make mistakes in public, and then count on the public
to correct those mistakes. This does indeed give the mathematics community
powers to discern mathematical truth that dwarf the powers of any individual
human brain (even an individual brain with paper-and-pencil peripherals, a
hand calculator, or a laptop!). But this does not show that human minds are
not algorithmic devices; on the contrary, it shows how the cranes of culture
can exploit human brains in distributed algorithmic processes that have no
discernible limits.
Penrose doesn't quite see it that way. He goes on to say that "it is our
general (non-algorithmic) ability to understand" that accounts for our
mathematical abilities, and then he concludes: "It was not an algorithm x
that was favoured, in Man ( at least) by natural selection, but this wonderful
ability to understand!" (Penrose 1991). Here he commits the fallacy I just
exposed using the chess example. Penrose wants to argue:
x can understand;
there is no feasible algorithm for understanding;
therefore: what natural selection selected, the whatever-it-is that accounts
for understanding, is not an algorithm.
This conclusion is a non sequitur. If the mind is an algorithm (contrary to
Penrose's claim), surely it is not an algorithm that is recognizable to, or
accessible to, those whose minds it creates. It is, in his terms, unknowable.
As a product of biological design processes ( both genetic and individual), it
is almost certainly one of those algorithms that are somewhere or other in the
Vast space of interesting algorithms, full of typographical errors or "bugs,"
but good enough to bet your life on—so far. Penrose sees this as a "farfetched"
possibility, but if that is all he can say against it, he has not yet
come to grips with the best version of "strong AI."