Notes for the Lifebox, the Seashell, and the Soul - Rudy Rucker

Notes for The Lifebox, the Seashell, and the Soul, by Rudy Rucker
Back-propagation Momentum Term
As I mentioned in passing in the last chapter, when training a neural net, we also
avoid getting stuck on local maxima by using a so-called momentum term which keeps us
moving forward through the fitness landscape even when a maximum is found. If no better
maximum occurs nearby, the momentum peters out and the search circles back.
Game AI
There’s a variety of ways one might approach this kind of problem. The so-called
expert system approach is to analyze how a good football player avoids tacklers ⎯ when to
fake, when to speed up, when to slow down, when to double back, and so on. With enough
trial and error, you can code up an intricate reflex schema as a series of if-then rules, perhaps
of the form:
if Condition1 then do Action1;
if Condition2 then do Action2;
if Condition3 then do Action3;
if Condition4 then do Action4;
...
if Condition5 then do Action5;
With sufficient care, you could make the conditions both mutually exclusive and
complete ⎯ that is, you could define the conditions so that at any given time exactly one of
them would hold. But this isn’t always the best way to go. At some point it might be
although Condition1 is true, Condition2 is very nearly true, so that the best thing might be
to mix a bit of reaction Action2 in with your reaction Action1.
How would I mix two actions? Some kinds of actions lend themselves to being
mixed. If Action1 is to turn left by ten degrees, and Action2 is to turn left by thirty degrees,
then one might possibly combine the two by turning left twenty degrees. But this could be a
mistake. If, say, Action1 is to turn left ten degrees and Action2 is to turn right ten degrees,
does combining them mean that you shouldn’t turn at all? In some situations this might be
exactly the wrong move. A different way to combine actions is probabilistically, as if by
rolling weighted die. This is in fact the only option if the actions are incompatible, as when,
for instance, Action1 is passing the ball and Action2 means kicking the ball.
To implement a mixed approach of any kind, rather than assuming that each condition
is definitely true or false, we can instead give the truth of each condition a weight between 0
and 1. 0 is false, 1 is true, 0.2 is almost false, 0.9 is very nearly true, and so on.
This sounds nebulous, but we can make it precise. Suppose, for instance, Condition
expresses the notion “the left guard is too close to me”. If I let distance stand for the
distance between me and the opposing team’s left guard, then I might be comfortable in
giving Condition a value of 0 if distance is greater than 10 yards, and giving Condition a
value of 1 if distance is less than 1 yard. And maybe for the in-between distances I could use
a linear expression of the form (1 - distance/10), or if I preferred, a more complicated
expression of like (1 - distance2/100)
Once we got the truth values for the various conditions we could use them in one of
two ways, as coefficients in a weighted sum or as probabilities in a random die.
p. 74