The Development of Neural Network Based System Identification ...

82 CHAPTER 4 NEURAL NETWORK BASED SYSTEM IDENTIFICATION
f(x)
6
5
4
3
2
1
0
-6 -5 -4 -3 -2 -1 -1 0 1 2 3 4 5 6
-2
-3
-4
-5
-6
x
(a) f(x) = x
f(x)
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
-6 -4 -2 0 2 4 6
x
(b) f(x) = 1
1+e −x
f(x)
1.2
1
0.8
0.6
0.4
0.2
0
-6 -4 -2 -0.2 0 2 4 6
-0.4
-0.6
-0.8
-1
-1.2
x
(c) f(x) = e2x −1
e 2x +1
f(x)
1.1
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
x
(d) f(x) =
{ 1 if x > 0
0 if x < 0
1.1 f(x)
0.9
0.7
0.5
0.3
0.1
-6 -5 -4 -3 -2 -1
-0.1
0 1 2 3 4 5 6
-0.3
-0.5
-0.7
-0.9
-1.1
x
⎧
⎨
(e) f(x) =
⎩
1 if x > 0
0 if x = 0
−1 if x < 0
f(x)
1.2
1
0.8
0.6
0.4
0.2
0
-4 -3 -2 -1 0 1 2 3 4
-0.2
x
(f) f(x) = ae − (x−b)2
2c 2
Figure 4.3 Different types of activation function for NN modelling: (a) Linear function; (b) Sigmoid
function; (c) Hyperbolic Tangent function; (d) Step; (e) Sign function; and, (f) Gaussian function with
real constant a, b, c > 0
flow [Wilamowski, 2011b]. Typically, MLP networks are constructed with an input
layer, one or more hidden layers and an output layer. These three layers are linked
by weights connection resulting in two set of weights, the input-hidden layer weight
and the hidden-output layer weight. The NN can be constructed with more hidden
layers, however, there are no significant advantages or practical reasons in including