The Development of Neural Network Based System Identification ...

148 CHAPTER 5 NN BASED SYSTEM IDENTIFICATION: RESULTS AND DISCUSSION
essential since the control decisions need to be updated at the specific timing requirement
(22 ms). There are two types of recursive algorithm methods that exist to approximate
the non-linear dynamics in real-time; a) mini-batch methods [Samal, 2009, Puttige,
2009], and b) recursive prediction method as presented in previous Section 4.3.5. For
mini-batch wise methods, off-line training such as LM algorithm was used to train
the NN in real-time by choosing smaller data length to achieve faster convergence
time.
Typically, a fixed amount of input-output data is collected and stored in a
queue. Table 5.11 shows the average training time for mini-batch LM and rGN training
algorithms for attitude dynamics identification using the optimal MLP model structure
(n y = 3, n u = 1 with 4 hidden neurons). The minimum criterion error (MSE) was
selected at 0.001, 0.01 and 0.05 as stopping criteria for mini-batch LM training. The
training time comparison test was conducted using a 400 MHz National Instrument’s
real-time embedded controller. Result from the comparison test shows that mini-batch
LM training produces faster training convergence with smaller batch sizes. However,
mini-batch LM method requires high computation resources and would not finish within
the targeted control sampling period (22 ms). Attempts to reduce the training time
of the NN training through manipulation of target MSE values could improve the
algorithm training performance, but at the expense of poor training error. A recursive
training algorithm such as rGN usually demonstrates faster prediction updates and
offers rapid computation of weight adaptation with average training time of 3.88 ms.
The average training time for rGN algorithm is well below the control loop sampling
period (22 ms) and this indicates that such recursive training algorithms are well suited
for real-time applications.