The Development of Neural Network Based System Identification ...

5.8 SUMMARY 151
methods such as weight pruning or introducing regularisation term into the LM training
algorithm. For system identification using the off-line LM training, results show training
with regularisation term introduces a smoothing effect on the error criterion V N (θ, Z N )
in such a way that weights that have less important influence on error are forced to
decay towards zero [Samarasinghe, 2007]. In this process, only the important weights
that minimise the error are allowed to grow and stabilise at their optimum values. The
implementation of regularisation term during NN training prevents the trained NN
model from over-fitting that would occur when the NN model is presented with a new
test data set.
In order to get a better prediction performance from the NN model, the model
structure of the NN model can be further improved through proper network structure
selection. To select the NN model structure, the Lipschitz coefficient calculation and
k-cross-validation test methods were proposed, and they were used to identify the
optimal or near optimal model structure of the NN model without attempting the
tedious trial and error approach. The validation result shows that the model structure
of the MLP architecture can be identified correctly with 3 past outputs and 1 past
input using the proposed methods. Further comparison with model structure selection
from previous studies such as in Samal [2009] and Putro et al. [2009] show that the
identified model structure using k-cross validation offers an improvement in terms of
generalisation error. Similarly, the minimum number of neurons to be included in the
NN model can also be selected using the proposed validation methods.
The HMLP and modified Elman networks were proposed in this work to provide
us with simpler representation of UAS dynamics and reduction in total number of
weight connections used in the NN model. Furthermore, the reduction in the total
number of weights in the network can significantly reduce the computation time needed
to train the NN model. Similar methodologies to select the optimal structure in the
MLP network was used to identify the network structure for both HMLP and modified
Elman networks. Based on validation test results, the model structures of the HMLP
and modified Elman networks are found to be much smaller than the standard MLP
network. Although the total number of weights for the HMLP and modified Elman