The Development of Neural Network Based System Identification ...

134 CHAPTER 5 NN BASED SYSTEM IDENTIFICATION: RESULTS AND DISCUSSION
RMSE (%)
28
26
24
22
20
18
16
14
h=1
h=2
h=3
h=4
h=5
h=6
h=7
h=8
12
10
8
6
1−1 (4 regs.) 2−1 (6 regs.) 2−2 (8 regs.) 3−1 (8 regs.) 3−2 (10 regs.) 3−3 (12 regs.)
Lag Space
Figure 5.10 The percentage of Root Mean Square Error (RMSE) of the HMLP network trained with
different network structures and number of hidden neurons. The neural network training was carried
out using off-line Levenberg-Marquardt (LM) algorithm.
Table 5.4
The HMLP neural networks model parameters.
HMLP Network Specifications for Attitude Dynamics
Number of past outputs 2
Number of past inputs 1
Number of neurons in hidden layer 3
Activation function at hidden layer
Tanh
Activation function at output layer
Linear
Number of regressors 6
Total number of weights 43
Weight decay 0.0001
plots of the optimum HMLP structure, the network is found capable of producing RMSE
as good as the standard MLP network at much lower number of hidden neurons and
network structure. This suggests that the additional linear connections from the input
layer to output layer in the HMLP network helps reduces the complexity of the MLP
network by incorporating linear weights connections instead of adding more neurons in
the hidden layer. Furthermore, the signal propagations in the linear weight connections
across layers are easier to train compared with signals that pass through non-linear
neurons [Wilamowski, 2009]. Thus, the reduced network complexity in the HMLP
network can leads to a faster learning rate such as demonstrated in Section 5.7.