AI - a Guide to Intelligent Systems.pdf - Member of EEPIS

THE HOPFIELD NETWORK
195
The Hopfield network will always converge to a stable state if the retrieval is
done asynchronously (Haykin, 1999). However, this stable state does not
necessarily represent one of the fundamental memories, and if it is a fundamental
memory it is not necessarily the closest one.
Suppose, for example, we wish to store three fundamental memories in the
five-neuron Hopfield network:
X 1 ¼ðþ1; þ1; þ1; þ1; þ1Þ
X 2 ¼ðþ1; 1; þ1; 1; þ1Þ
X 3 ¼ð 1; þ1; 1; þ1; 1Þ
The weight matrix is constructed from Eq. (6.20),
2
W ¼
6
4
3
0 1 3 1 3
1 0 1 3 1
3 1 0 1 3
7
1 3 1 0 1 5
3 1 3 1 0
Assume now that the probe vector is represented by
X ¼ðþ1; þ1;
1; þ1; þ1Þ
If we compare this probe with the fundamental memory X 1 , we find that these
two vectors differ only in a single bit. Thus, we may expect that the probe X will
converge to the fundamental memory X 1 . However, when we apply the Hopfield
network training algorithm described above, we obtain a different result. The
pattern produced by the network recalls the memory X 3 , a false memory.
This example reveals one of the problems inherent to the Hopfield network.
Another problem is the storage capacity, or the largest number of fundamental
memories that can be stored and retrieved correctly. Hopfield showed
experimentally (Hopfield, 1982) that the maximum number of fundamental
memories M max that can be stored in the n-neuron recurrent network is
limited by
M max ¼ 0:15n
ð6:26Þ
We also may define the storage capacity of a Hopfield network on the basis
that most of the fundamental memories are to be retrieved perfectly (Amit,
1989):
M max ¼
n
2lnn
ð6:27Þ