The Development of Neural Network Based System Identification ...

4.3 SYSTEM IDENTIFICATION WITH NEURAL NETWORK 107
layer. By performing differentiation as in MLP example, the Jacobian matrix for HMLP
is given as:
⎧
v h (t)
if θ = W 2 ih
ϕ j (t)
if θ = W 3 ij
ψ (t|θ) hmlp
=
∂ŷ (t|θ)
∂θ
⎪⎨
=
1 if θ = B2 i
(
W 2 ih 1 − v
2
h
(t) ) ϕ j (t)
(
W 2 ih 1 − v
2
h
(t) )
if θ = W 1 hj
if θ = B1 h
(4.39)
⎪⎩
0 otherwise
Similarly, the output prediction from the modified Elman network can also be
expressed with the hyperbolic tangent hidden units and linear output unit as follows:
⎛
⎞
m∑
h∑
v h (t) = tanh ⎝ W 1 hj ϕ j (t) + B1 h + W 3 k x k (t) ⎠
j=1
k=1
x k (t) = v h (t − 1) + αx k (t − 1)
H∑
ŷ i (t |θ ) Elman
= W 2 ih v h (t) + B2 i
h=1
with h = 1, 2, 3 · · · H and i = 1, 2, 3 · · · n (4.40)
where W 3 k is the weights vector for connections between context units and hidden
layer. The vector x k (t) denotes the output of the context units. By performing the
differentiation of network output with respect to each weight term, the Jacobian matrix
for Elman network is given as follows:
⎧
v h (t)
if θ = W 2 ih
W 2 ih
(
1 − v
2
h
(t) ) ϕ j (t)
if θ = W 1 hj
ψ (t|θ) Elman
=
∂ŷ (t|θ)
∂θ
⎪⎨ W 2 ih
=
∂v h (t)
∂W 3 k (t−1)
if θ = W 3 k
1 if θ = B2 i
W 2 ih
(
1 − v
2
h
(t) )
if θ = B1 h
(4.41)
⎪⎩
0 otherwise