Chapter 2. Prehension

Appendix C - Computational Neural Modelling 395
In order to use gradient descent, a continuous, nonlinear activation
functionf is needed, since the derivative is used and because hidden
units are part of the network topology. The disadvantage of this
technique is that a local minimum could be reached which is not a true
minimum. There is also no indication of how long multiple repetitions
of presenting the inpudoutput pairs are needed in order to get the error
reduced to some small value close to zero.
C.2.3 Processing capabilities in neural networks
Some of the types of information processing performed by
artificial neural networks include pattern association, pattern
completion, pattern classification, and regularity discovery. In pattern
association, a pattern is not stored; instead, the strength of the
connection between two neurons adjusts, so that the next time an input
pattern is seen at neuron j that causes it to output Oj, the weight Wij has
been set to the correct value to produce the associated pattern Oi as
output from neuron i. Pattern completion involves filling in a missing
portion of a pattern. For example, a pattern is associated with itself, in
what is called auto-association, so that when a degraded version of the
pattern is presented to the network, it can reconstruct the complete
pattern. Degraded patterns occur when incomplete information is
available, such as in partially-hidden objects, or noise on a signal.
Particularly useful properties of artificial neural networks include
default assignments, generalization, and graceful degradation. A
computational architecture that can generalize about how to grasp
objects never seen before is better than a system where every possible
associated pattern must be accounted for. In these artificial neural
networks, where similar input patterns lead to similar outputs, only a
few sample training points are needed. Similar patterns reinforce the
strengths of the weights between neurons. By presenting a pair with
some noise added, the system will learn the central tendency. Default
assignments as well come from this same property. Also, one does
not need a perfect copy of the input to produce the output.
C.3 Network Example: Heteroassociative
Memories
A single-layer, feedforward neural network may be used as a
heteroassociative memory. That is, it can be trained to produce an
arbitrary output pattern given a paired input pattern. We consider the
case where the network has an n-vector output of linear summation