Chapter 2. Prehension

388 A pp e n dices
processing. This in effect creates a higher dimensional input space,
allowing more ‘elbow room’ for the computation. It is often the case
that these hidden units attend to features of the input patterns important
to the network’s computation, so experimenters analyze their
responses to determine how the network operates.
A representation can either be encoded or decoded. With a
decoded input representation, there is a separate unit for each possible
input pattern. Since only one of these is active at a time, the network
must see every possible input pattern during training. The
disadvantage is that there is a combinatorially explosive number of
units. However, the weight modification rule is then simple, since
input units can drive output units with excitatory synapses, and inhibit
others. Other advantages of using highly decoded representations
include fault tolerance, sufficiency of components with relatively poor
dynamic range, and ease of local feature extraction. The ease of
learning a function will depend on the degree to which the output
representation is related to the input representation on which it
depends. With an encoded input representation, the activation of
output units depends on complex nonlinear interactions between the
inputs, and therefore multiple layers for computing more complex
functions are needed.
Inputs I
+ t
outputs
*
rA
a
P
rA
c)
a
P
c)
f 8
c(
Figure C.4 Recurrent networks that are completely connected.
Both networks receive input directly from every other unit.
The alternative to feedforward networks is the recurrent or
feedback network, as seen in Figure C.4. These examples are both
completely connected recurrent networks; i.e., every unit receives