The Development of Neural Network Based System Identification ...

102 CHAPTER 4 NEURAL NETWORK BASED SYSTEM IDENTIFICATION
4.3.4 Off-line based Neural Network Model Estimation
After selecting the model structure, the next step in the system identification process is
to determine the best weights (vector parameters θ) that give the best fit between the
NNARX model and measurement data. This is achieved by minimisation of error cost
function. As mentioned in Norgaard [2000], the measure of prediction’s closeness to the
true outputs of the system is given by mean square error type criterion as:
V N (θ, Z N ) = 1
2N
N∑
t=1
[y (t) − ŷ (t |θ )] 2 = 1
2N
with linear approximation of prediction error given by:
N∑
e 2 (t, θ) (4.28)
t=1
e (t, θ) = y (t) − ŷ (t |θ ) (4.29)
where N is the number of input-output pairs used for training, y(t) is the real measurement
output of the system, ŷ(t|θ) is the predicted output vector and the training
data set is given by Z N = [y(t), u(t)]. For multiple input-output case (n outputs), the
measurement output of the system y(t) and predicted output ŷ(t|θ) will became a n × N
matrix which would produced a vector of mean square error criterion.
The solution involving the quadratic criterion in Equation (4.28) is also known as
ordinary non-linear least squares problem which is a part of unconstrained optimisation
study [Norgaard, 2000, Sofer et al., 2009]. The minimisation of criterion is carried out
using numerical search procedure starting out from initial guess of parameter vector
θ (0) . The weights of the network are then adjusted according to some training methods
and stopped after error criterion evaluation achieves certain threshold. Typically, most
off-line minimisation iterative schemes have the following general form:
[
θ (i+1) = θ (i) − µ (i) H (i)] −1 )
′
V N
(θ (i) , Z N
(4.30)
where θ (i) is the parameter vector estimation at i th iteration, H (i) is the matrix that
(
modifies the local search direction defined by V ′ N θ (i) )
, Z N and constant µ (i) denotes
(
the step size to assure that V N θ (i) )
, Z N decreases from previous update.