The Development of Neural Network Based System Identification ...

4.2 THE ARTIFICIAL NEURAL NETWORKS 85
ŷ1
yˆn
1
h
MLP connection
B
Bias Input
Linear connection
X1
X2
Xm
b
Bias Input
Figure 4.5 The hybrid multi-layered perceptron (HMLP). The dashed lines indicate extra linear
connections in the network.
extra linear connections indicated in the figure. Mashor [2000] suggested that HMLP
network structure offers improved training efficiency and generalisation properties for
neural network modelling. Furthermore, the utilisation of linear connection in HMLP
can significantly reduce the number of neurons used in the hidden layer as it allows
connections between all layers [Hunter and Wilamowski, 2011, Wilamowski et al., 2008].
The HMLP architecture consists of input, hidden and output layer with the same
functions as the one in standard MLP architecture.
Each node in input layers is
connected to each node in hidden and output layer, and each node in hidden layer is
connected to output layer forming a fully connected network. Since HMLP is a variant
of MLP architecture, the network architecture can be constructed using multiple hidden
layers; however there are usually no significant advantages or improvements on model
prediction performance [Tu, 1996, Samal, 2009]. For a single hidden layer case, the
outputs formulation from hidden and output layer of an HMLP network is given as
follows:
v h (t) = g h
⎛
⎝
⎞
m∑
W 1 hj X j (t) + B1 h
⎠ for h = 1, 2, 3 · · · H (4.6)
j=1