The Development of Neural Network Based System Identification ...

124 CHAPTER 5 NN BASED SYSTEM IDENTIFICATION: RESULTS AND DISCUSSION
in Figure 5.3(c) shows that the weights gradually increase and remain constant beyond
iteration point i = 185. The network training ends much quicker compared with the
network training without regularisation term since the weights do not change after
i = 185. The MSE calculation on the validation data set also shows that the optimum
weights are found at i = 185, thus demonstrating the effectiveness of the regularisation
method to prevent the over-fitting problem. The effect of using the regularisation term
basically introduces a smoothing effect on the error criterion V N (θ, Z N ) in such a way
that weights that have less important influence on error are forced to decay towards
zero [Samarasinghe, 2007]. In this process, only the important weights that minimise
the error are allowed to grow and stabilise at their optimum values.
The MSE result for network training with large regularisation parameter α = 0.8
is shown in Figure 5.3(f). The figure indicates that the network training stops much
earlier than the network training with α = 0 and α = 0.0001. However, the network
model has higher MSE values in both training and validation data. Further increase in
regularisation parameter would make the weights less adaptable as the weights only
grow in limited small magnitude (±1.5). This would make the network less flexible and
would result in poor prediction performance due to severe bias.
5.2.2 Model Structure Selection Results
The optimum model structure can be found using the Lipschitz coefficient, and it is
possible to determine the proper lag space via experimental data [Norgaard, 2000].
The result of the Lipschitz coefficient calculation for a pair of input and output data
(δ lat and p) is shown in Figure 5.4(a). It is shown that the Lipschitz coefficient curve
decreases and stabilises at n y = 3 and n u = 1 for that particular pair of input and
output data selection. For this input-output pair, the reasonable network structure to
describe the attitude dynamic (p/δlat) is by selecting a number of past outputs n y = 3
and number of past inputs n u = 1. By summing all the stabilising points for each data
pair, a total number of 8 time regressors are fed to the neural network. Finally, the
selected neural network based on the ARX model (NNARX) structure to identify the
non-linear relationship of helicopter’s attitude dynamics is shown in Figure 5.4(b).