The Development of Neural Network Based System Identification ...

126 CHAPTER 5 NN BASED SYSTEM IDENTIFICATION: RESULTS AND DISCUSSION
coefficient is calculated on noisy data, the coefficient index plot would not exhibit a
sharp breakpoint before stabilising at a large model order region. This would lead to
incorrect model order selection as network designers would probably select a higher
model order in the smooth region. To validate whether NN model structure with 3 past
outputs and 1 past inputs represented the correct model structure for the underlying
dynamics, the k-fold cross validation was carried out next to determine the best network
structure.
Different network structures found from past research works such as Putro et al.
[2009] and Samal [2009] were used for comparison to determine the best network
structure using k-fold validation method. In this study, the flight data obtained from
the experiment was divided into 10 approximately equal segments. In the validation
stage, the error calculation was then stored for every network structure and hidden
neuron case. Subsequently, the stored error calculation was then retrieved at the end of
the validation cycle for RMSE computation.
The result of k-cross validation for different network structures is given in Figure 5.5.
Six different network structures were tested and compared with each other. The plot
indicates that network structure with 1 past output and 1 past input (4 regressors) gives
the highest percentage of RMSE. This indicates that a simple network structure with
1 past output and 1 past input (4 regressors) is not suitable to be used for predicting
the non-linear dynamic system. As the number of regressors or inputs to the network
increases, the RMSE value decreases and stabilised after 3 past outputs and 1 input
(8 regressors) structure. Hence, the neural network model structure can be selected
as a total of 8 regressors with 3 past outputs and 1 past input. This cross validation
procedure was repeated for different hidden neuron sizes and an overall RMSE trend
points to the same sharp breakpoint at 3 past outputs and 1 input (8 regressors) network
structure. This further indicates that the generalisation performance of the network is
more sensitive to the effect of network structures rather than hidden neurons.
The optimum number of hidden neurons used in the network was also determined
using k-fold cross validation method. The simulation result of hidden neuron selection
for 3 past outputs and 1 past input network structure is given in Figure 5.6. From the