The Development of Neural Network Based System Identification ...

4.3 SYSTEM IDENTIFICATION WITH NEURAL NETWORK 111
4.3.5 Recursive based Neural Network Model Estimation
The recursive system identification method builds a model of the system at the same
time as the measurement data is collected. The prediction model is then updated
at each time step, as new data become available. In our study, the weight updating
procedure is calculated using recursive Gauss Newton (rGN) method. It was originally
derived by Ljung and Soderstrom [1983] and later on modified in Chen et al. [1990] to
train the MLP network. For every data sample, the parameter vector ˆθ (t) is updated
by the recursive algorithm using the following equations:
e (t) = y (t) − ŷ (t) (4.48)
R (t) = λ(t)R (t − 1) + (1 − λ(t)) ψ (t) ˆΛ −1 (t) ψ T (t) (4.49)
K (t) = (1 − λ(t)) R −1 (t) ψ (t) ˆΛ −1 (t) (4.50)
ˆθ (t) = ˆθ (t − 1) + K (t) e (t) (4.51)
where R(t) is an approximation of the Gauss-Newton Hessian matrix, ˆθ(t) is the
estimation of parameters vector of the neural network model, ˆΛ −1 (t) is the weighting
matrix and λ(t) denotes the forgetting factor at the current time step t. The simplest
choice of weighting matrix ˆΛ −1 (t) is an identity matrix as suggested by Billings et al.
[1992]. The forgetting factor λ(t) is defined as a constant scalar variable which accounts
for the amount of past data information to be included in the error criterion function.
If the forgetting factor is λ(t) < 1, the term would make the estimation more adaptable
to changes and sensitive to noise. Whereas, if λ(t) → 1 as time increases, more old data
are included in the criterion and the adaptation would fluctuate less during the learning
process [Youmin and Li, 1999].
In practice, Equation (4.48)-(4.51) are not calculated straightforward with inversion
of matrix R −1 (t) which requires computational complexity of O(d 3 ) [Ngia and Sjoberg,
2000]. Ljung and Soderstrom [1983] had shown that the matrix inversion complexity
can be reduced from complexity of O(d 3 ) to O(d 2 ) using matrix inversion theorem as
follows:
[A + BCD] −1 = A −1 B [ DA −1 B + C −1] −1
DA
−1
(4.52)