AI - a Guide to Intelligent Systems.pdf - Member of EEPIS

MULTILAYER NEURAL NETWORKS
179
k ðpÞ ¼y k ðpÞ½1 y k ðpÞŠ e k ðpÞ; ð6:14Þ
where
y k ðpÞ ¼
1
1 þ exp½ X k ðpÞŠ :
How can we determine the weight correction for a neuron in the hidden
layer?
To calculate the weight correction for the hidden layer, we can apply the same
equation as for the output layer:
w ij ðpÞ ¼ x i ðpÞ j ðpÞ;
ð6:15Þ
where j ðpÞ represents the error gradient at neuron j in the hidden layer:
j ðpÞ ¼y j ðpÞ½1
y j ðpÞŠ Xl
k¼1
k ðpÞw jk ðpÞ;
where l is the number of neurons in the output layer;
y j ðpÞ ¼
X j ðpÞ ¼ Xn
1
1 þ e X jð pÞ ;
i¼1
x i ðpÞw ij ðpÞ j ;
and n is the number of neurons in the input layer.
Now we can derive the back-propagation training algorithm.
Step 1:
Initialisation
Set all the weights and threshold levels of the network to random
numbers uniformly distributed inside a small range (Haykin, 1999):
2:4
; þ 2:4
;
F i F i
where F i is the total number of inputs of neuron i in the network. The
weight initialisation is done on a neuron-by-neuron basis.
Step 2:
Activation
Activate the back-propagation neural network by applying inputs
x 1 ðpÞ; x 2 ðpÞ; ...; x n ðpÞ and desired outputs y d;1 ðpÞ; y d;2 ðpÞ; ...; y d;n ðpÞ.
(a)
Calculate the actual outputs of the neurons in the hidden layer:
" #
y j ðpÞ ¼sigmoid Xn
x i ðpÞw ij ðpÞ ;
i¼1
j
where n is the number of inputs of neuron j in the hidden layer,
and sigmoid is the sigmoid activation function.