Notes for the Lifebox, the Seashell, and the Soul - Rudy Rucker

Notes for The Lifebox, the Seashell, and the Soul, by Rudy Rucker
triples of real numbers, with all three coordinates ranging between minus one and plus one.
You’re searching through, in other words, a three-dimensional cube centered on the origin,
and you want to find the “fittest” point. Suppose that you let the distance between two
solutions be simply the usual Euclidean notion of distance. A conceivable “annealing
schedule” might go like this.
• Take a population of 64 hill-climbers and set them down at random locations in the
solution space. Set the temperature to 2.
• At each update, have each hill-climber evaluate the fitness at its present location and at
eight randomly chosen locations that are within a distance of T from its current location.
Move the hill-climber to the location among the nine points that has the highest fitness.
• Every ten updates reduce T by 0.1 until it reaches 0.1, and then begin reducing it by 0.01
every ten updates, and when it reaches 0.01 begin reducing it by 0.001 every ten updates.
• When T reaches 0.001 return the solution specified by the most successful of your hillclimbers.
Choosing the right annealing schedule for a given solution space is of course a
metasearch problem of its own. Systematizing search processes is perhaps as much a craft as
it is a science.
Self-Deprecating Remark Introducing Chapter 4
When all else fails, systematize!
Continuous Response
In the world of living organisms, we rather expect the inputs and outputs to be
smooth, rather than abrupt zeroes and ones. If something is only a little bit hot, for instance,
you might move your finger only a little bit.
Sigmoid Neurons
There are various ways to set up these artificial neurons; I’ll describe one commonly
used approach. We’ll suppose that one of our neurons can output a real number ranging from
zero to one, and we’ll suppose that it calculates this output by summing up some number of
input signals with a particular weight being multiplied times each input signal. Unlike the
signals, the weights are allowed to be negative, we let them lie between negative one and
positive one. 6 We might think of negatively weighted inputs as inhibiting our artificial
6 As always, when we talk about continuous numbers inside digital machines we need to
pause and say that in practice the real numbers used by a PC aren’t truly continuous, they’re digital
models of continuous numbers. As mentioned before, if we take the common expedient of
representing a real number by thirty-two bits, then we’re really allowing for “only” four billion
different real numbers.
I also should mention that in practice, biological neurons do not have continuous-valued
outputs. A brain neuron either fires or it doesn’t fire; its output is more like a single-bit 0 or 1.
Computer scientists prefer model neurons with continuous-valued ouputs because it’s easier to tailor a
smallish network of output-valued-output neurons to perform a task. If you insist on binary-valuedp.
71