The Development of Neural Network Based System Identification ...

96 CHAPTER 4 NEURAL NETWORK BASED SYSTEM IDENTIFICATION
v(t) [Ljung, 1999]. It is a well known type of system model and also known as linear
regression in statistics.
Since we are considering a system that is non-linear in nature, the non-linear term
h(.) can be introduced to the linear ARX model predictor and the resulting non-linear
ARX model structure with k-step predictor is denoted as:
ŷ (t + k |t, θ ) = h [ ϕ T (t + k)θ ] (4.23)
Parameter vector θ in Equation (4.23) contains adjustable parameters which can also
be represented in the neural network model as weight connections. The processing
neurons in the hidden layer allow the NN model to learn the non-linear relationship
between the measured outputs and inputs. The NN representation of the non-linear
ARX model structure is known as Neural Network ARX (NNARX) and it is also known
as Serial-Parallel model or focus time lagged feed-forward network in NN community
[Samal, 2009, Samarasinghe, 2007].
The fully connected MLP or HMLP architecture containing only one hidden layer
discussed in Section 4.2.1 and 4.2.2 can be chosen to learn the non-linear relationship of
the NNARX model. The conceptual diagram of NNARX model structure using MLP or
HMLP network are given in Figure 4.9(a) and 4.9(b). The output calculation from the
MLP structure in Equation (4.1) and (4.2) is reproduced here to represent the NNARX
predictor formulation for a single hidden layer MLP network:
⎛ ⎛
H∑
ŷ (t |θ ) = h i (ϕ, θ) = F i
⎝ W ih f h
⎝
h=1
m∑
j=1
⎞ ⎞
w hj ϕ j (t) + b h
⎠ + B i
⎠
with h = 1, 2, 3 · · · H and i = 1, 2, 3 · · · n (4.24)
and the parameter and regression vectors are given by:
θ = [ w hj W ih b h B n
]
(4.25)