The Development of Neural Network Based System Identification ...

100 CHAPTER 4 NEURAL NETWORK BASED SYSTEM IDENTIFICATION
model estimation is a system identification technique that enables us to infer a model
that adapts to time-varying dynamics based on real-time data coming from the system.
In contrast to batch training methods, the recursive methods enforce update to NN
parameter vector based only on a single data set at the current sample t. To achieve
real-time implementation of neural network based system identification, the estimation
of neural network’s parameter vector θ can be carried out using recursive algorithms as
described in Billings et al. [1992], Norgaard [2000], Youmin and Li [1999], Ljung and
Soderstrom [1983], Asirvadam [2008], Shamsudin and Chen [2012b]. Norgaard [2000] has
suggested that the recursive identification algorithms have several advantages over the
batch methods. The implementation of the method is simpler, less memory-consuming
with faster convergence since the redundancy in data set is effectively utilised.
Recursive algorithm can also be implemented similar to the off-line training, where
the recursive training is repeated several time on the finite training set Z N collected in
advance [Norgaard, 2000, Billings et al., 1991, 1992]. Figure 4.10 shows the difference
between the batch algorithm, mini-batch algorithm, on-line recursive algorithm and
repeated recursive algorithm. The parameter vector θ updating process usually starts
with initial random weights and it is carried out forward to the next iteration as computation
progresses. The implementation of batch and mini-batch algorithm is similar
but differs in the number of data samples used for training. In the recursive algorithm
methods, the parameter vector update is obtained in real-time as the measurement
data become available from the instrumentation system. Figure 4.10(d) shows the
implementation of recursive algorithm as an off-line method. The parameter vector
θ is updated at each time sample t over a fixed length data sample. At the end of
first iteration, the last parameter vector θ is used as the initial update to the second
iteration step. This iteration process will stopped if the mean square criterion converges
to pre-defined threshold as in batch training algorithm implementation.