AI - a Guide to Intelligent Systems.pdf - Member of EEPIS

WILL A NEURAL NETWORK WORK FOR MY PROBLEM?
325
Another problem is overfitting. The greater the number of hidden neurons, the
greater the ability of the network to recognise existing patterns. However,
if the number of hidden neurons is too big, the network might simply memorise
all training examples. This may prevent it from generalising, or producing correct
outputs when presented with data that was not used in training. For instance, the
overfitted character recognition network trained with Helvetica-font examples
might not be able to recognise the same characters in the Times New Roman font.
The practical approach to preventing overfitting is to choose the smallest
number of hidden neurons that yields good generalisation. Thus, at the starting
point, an experimental study could begin with as little as two neurons in the
hidden layer. In our example, we will examine the system’s performance with
2, 5, 10 and 20 hidden neurons and compare results.
The architecture of a neural network (with five neurons in the hidden layer)
for the character recognition problem is shown in Figure 9.20. Neurons in the
hidden and output layers use a sigmoid activation function. The neural network
is trained with the back-propagation algorithm with momentum; the momentum
constant is set to 0.95. The input and output training patterns are shown in
Table 9.2. The binary input vectors representing the bit maps of the respective
digits are fed directly into the network.
The network’s performance in our study is measured by the sum of squared
errors. Figure 9.21 demonstrates the results; as can be seen from Figure 9.21(a),
a neural network with two neurons in the hidden layer cannot converge to a
solution, while the networks with 5, 10 and 20 hidden neurons learn relatively
fast. In fact, they converge in less than 250 epochs (each epoch represents an
entire pass through all training examples). Also note that the network with 20
hidden neurons shows the fastest convergence.
Once the training is complete, we must test the network with a set of test
examples to see how well it performs.
Figure 9.20
Neural network for printed digit recognition