Notes for the Lifebox, the Seashell, and the Soul - Rudy Rucker

Notes for The Lifebox, the Seashell, and the Soul, by Rudy Rucker
modifying regimen such as meditation classes or group therapy. A pathological neurosis can
be so deeply ingrained a system bug that one has to dig down quite far to change it.
Good News Bad News
Depending on how you look at it, this may seem like either good news (we won’t be
replaced by smart robots) or bad news (we won’t figure out how to build smart robots).
Back Propagation as a Computation
In a way, a neural net is something very simple. The net-training process itself has a
simple and deterministic description: pick a network architecture, pick a pseudorandomizer
and assign starting weights, and do back-propagation training on a sample set. The only
complicated part here is the details of what lies in the sample set examples.
***
Again thinking back to evolution, you might wonder why we don’t tune a neural net
by using a genetic algorithm approach. You could look at a whole bunch of randomly
selected weight sets, repeatedly replacing the less successful weight sets by mutations and
combinations of the more successful weight sets.
You could do this, but in practice the neural net fitness landscapes are smooth enough
that the back-propagation hill-climbing method works quite well ⎯ and its faster and
simpler. But what about local maxima? Mightn’t a hill-climbing method end up on the top
of a foothill rather than on the top of a mountain? In practice this tends not to happen,
largely because so many weights are involved in a typical real-world neural net. Each
additional weight to tweak adds another dimension to the fitness landscape, and these extra
dimensions can act like “ridges” that lead towards the sought-for global maximum, sloping
up from a location that’s maximal for some but not all of the weight dimensions. Also, as
mentioned in the last chapter, we don’t really need absolute optimality. Reasonably good
performance is often enough.
Technical Problem with Running BZ on a Network Instead of on a CA
A network doesn’t have the neighborhood structure of a CA, like if A connects to B
and A connects to C, that doesn’t imply B is “near” C on a network, though it does in space?
Well, actually, if A will promise to act as a relay station, then B and C would be near in that
sense.
To go more towards network, how about giving each cell A its nearest neighbors NA
plus one remote neighbor rA. And say we pick the remote neighbors so they aren’t near each
other. That is, if A and B are close then rA and rB aren’t close. Maybe there could be a nice
canonical way to do this. If I wanted to program this, I could have a CA with an extra real
number field dist, and it would take the usual nearest neighbor inputs plus an input from a
cell whose index is dist*N where N is the size of the CA array. I could fill the dist fields at
startup with a randomizer. Oh, I did this years ago with John Walker, when I wrote the
ZipZap and XipXap series of CAs in assembly language. The remote neighbors didn’t
change much.
p. 73