Artificial Intelligence and Soft Computing: Behavioral ... - Arteimi.info

x1
x2
xn
14.6 Widrow-Hoff's Multi-layered
ADALINE Models
Another classical method for training a neural net with a sharp (hard limiting)
non-linearity was proposed by Widrow and Hoff in the 60's. The backpropagation
algorithm, which was devised in the 80's, however, is not
applicable to Widrow-Hoff's neural net because of discontinuity in the nonlinear
inhibiting function. Widrow-Hoff'’s proposed neuron is different from
the conventional neurons we use nowadays. The neuron, called ADALINE
(ADAptive LINEar combiner), consists of a forward path and a feedback loop.
Given a desired scalar signal dk , the ADALINE can adjust its weights using
the well-known Least Mean Square (LMS) algorithm. The signum type nonlinearity
on the forward path of the ADALINE prevents the signal level from
going beyond the prescribed limits. A typical ADALINE is shown in fig. 14.9.
For linear classification (vide fig. 14.10), where the entire space can be
classified into two distinct classes by a straight line, a single ADALINE
neuron is adequate. For example, to realize an 'AND' function by an
ADALINE, we can choose the weights such that w1 = w2 > 0 in the following
expression:
w1x1 +w2x2 > 0; x1, x2 ∈ {−1, +1 } (14.5)
w1
w2
wn
∆ ~
W = α ε
LMS Algorithm
2
k
n
∑
i=1
~
w xi
i
~
X /
k X k
Net
Fig. 14.9: A typical ADALINE neuron.
dk
+1
Net
–1
Out