Chapter 2 Introduction to Neural network

¡ ¢£ ¤¡¤¢¤£¥¦w x
4.2 The Linear Neuron (LMS-algorithm)
Is a neuron with a linear function
Assume we have a signal x(n) which we want to transform linearly
and match a signal d(n)
¡
x
¢£ ¤¡¤¢¤£¥¦w x
TDL L
We form the error signal e(n) = d(n) − y(n). We define the instantaneous
error function
§
J = 1 2 e(n)2
We want to minimize J. we use the steepest descent approach and
calculate ∇ w J
∇ w J = ∇ w
1
2 (d(n) − wT x(n)) 2
Used the chain rule and the equation become
∇ w J = (d(n) − w T x(n)) · (−x(n)) = −e(n) · x(n)
where the −x(n) is the inner derivative. The steepest descent states
w k+1 = w k − α∇ w J
w k+1 = w k + αe(n) · x(n)
(LMS-algorithm)
38