The Development of Neural Network Based System Identification ...

LIST OF FIGURES
xxi
5.2 The prediction performance result of a MLP network with 5 hidden
neurons trained with different weight decay values. The network was
trained 5 times using random weight initialisation. 122
5.3 The prediction performance of MLP network with varying regularisation
parameter α. (a), (c) and (e) show the weights adaptation during the
training process where each line represents single weight in the network.
The error evaluation on the training and validation datasets is shown
in (b), (d) and (f): (a) Network weights adaptation for α = 0 (no
regularisation term in the error cost function); (b) Training progress for
α = 0; (c) Network weights adaptation for α = 0.0001; (d) Training
progress for α = 0.0001; (e) Network weights adaptation for α = 0.8; and
(f) Training progress for α = 0.8. 123
5.4 Preliminary NN model structure selection from experimental input-output
data set using the Lipschitz coefficient (a) The Lipschitz coefficient plot
obtained for a pair of input and output data; and (b) The NNARX model
structure with preselected regression vectors obtained after determining
each individual Lipschitz coefficient from respective input-output pair. 125
5.5 The percentage of Root Mean Square Error (RMSE) of MLP network
model for each network structure and number of hidden neurons. The neural
network training was carried out using off-line Levenberg-Marquardt
(LM) algorithm. 127
5.6 The percentage of Root Mean Square Error (RMSE) comparison of MLP
network trained with different hidden neuron sizes. The k-cross validation
process was conducted for network structure with 8 regressors (n y = 3
and n u = 1). The neural network training was carried out using off-line
Levenberg-Marquardt (LM) algorithm. 128