Notes for the Lifebox, the Seashell, and the Soul - Rudy Rucker

Notes for The Lifebox, the Seashell, and the Soul, by Rudy Rucker
Incomprehensible Solutions
The neural net for recognizing smiles and frowns, for instance, is just a mound of
some three thousand real-valued numerical weights. You might hope that if you analyzed the
weights, you’d discover that each of the hidden layer neurons is in fact learning to recognize
a specific aspect of facial expressions. Perhaps the first neuron notices whether the left
corner of the mouth bends down or up, perhaps the second focuses on the wrinkles or lack
thereof in the face’s brow, and maybe the third pays attention to the twist of the mouth’s right
corner. But this isn’t necessarily the case. In many neural nets there’s no easy way to pick
out what it is that each individual hidden-layer neuron is learning. Indeed, the more compact
and effective a neural net becomes, the more opaque it becomes, and the less amenable to
human understanding. [Actually I don’t think this is usually true.]
Unsupervised Learning Raps
Let’s see what happens if we try to think of the human brain as being like a computer
scientist’s neural networks. One immediate thing to note is that, because a human is
embedded in the world, their brain network parameters are iteratively refined over the course
of years and years of parallel computational effort. Think of the finger movements of a
master musician or, for that matter, the empathetic understanding of an experienced teacher
or counselor.
It’s useful here to distinguish between two kinds of learning: supervised and
unsupervised.
Supervised learning is like when I train a neural net with a set of a hundred test faces.
I immediately tell the network which answers it got right and which it got wrong, and the
network back-propagates the errors to carry out a hill-climbing process to adjust its three
thousand or so internal weights. Supervised learning is like when you’re learning addition in
elementary school, and if you say “Five plus four is eight,” the teacher right away says, “No,
five plus four is nine,” and perhaps you internally adjust on some neural connections in your
brain.
But most of our learning is unsupervised in that the feedback is less immediate and
direct.
***
Unsupervised learning is when K goes out in the playground and a girl won’t get off
the seesaw to give him a turn, and K bites her on the butt, but she’s wearing thick snow pants
and doesn’t notice, but even so K feels bad and dumb. Unsupervised learning is when K
looks for someone to marry and ends up with the girl he tried to bite in first grade.
Unsupervised learning is when K finally tells her that old story and she divorces him. Error!
Unsupervised learning is when L drinks so much beer at age sixteen that she throws up,
learns to drink slowly enough not to get sick, but then realizes after thirty years of
increasingly vexed imbibing that she suffers from alcoholism. Unsupervised learning is M
dying of cancer from smoking cigarettes. Back-propagate that! Unsupervised learning is N
laboring in obscurity on a novel for fifteen years and then selling it and having a nice
success. N was right all along!
p. 75