The Development of Neural Network Based System Identification ...

98 CHAPTER 4 NEURAL NETWORK BASED SYSTEM IDENTIFICATION
and outputs of the neural network are presented by m and n respectively. Similarly,
the NNARX model can also be represented using HMLP network by introducing the
lagged time input variables into the neural network model.
The Elman network with modification in the network’s internal dynamics is found
capable of identifying an n th order discrete dynamic system in Equation (4.23), through
empirical findings in Pham and Liu [1993]. Furthermore, Pham and Liu [1996] suggested
that in order to model the dynamic system in Equation (4.23) from experimental data
using MLP network, a n y + n u input (regressor) nodes are needed to be included in
the network. However, if an Elman network is used to model a NNARX model, only
current measurement data are needed to be fed into the input nodes as shown in Figure
4.9(c). Therefore, the modified Elman network is significantly smaller in size compared
with MLP or HMLP network when large time lags (model order) are used.
4.3.2.1 Lag Space Selection for Feed-Forward MLP or HMLP Network
After deciding the input parameters to be used in the model, the number of past inputs
and outputs fed into the MLP or HMLP neural networks were decided based on the
calculation of the Lipschitz coefficient given in Norgaard [2000]. Using this coefficient,
it is possible to determine the proper lag space via experimental data. The sizes of
output and input time regression vectors depend on the degree of non-linearity of
the Lipschitz coefficients where an insufficient number of regressors will result in high
Lipschitz coefficients and small numerical values for extra regressors.
The Lipschitz coefficient is calculated using the following formula for each input
u i (t) and output y j (t) pairs:
q ij =
y (t i ) − y (t j )
∣ϕ (t i ) − ϕ (t j ) ∣ i ≠ j (4.27)
The outline of Lipschitz coefficient approach is given as follows:
1. Determine the Lipschitz quotients for all combination of input-output pairs using
(4.27) for a given choice of number of past outputs and inputs.