Neural Networks - Algorithms, Applications,and ... - Csbdu.in

9.2 Architectures of Spatiotemporal Networks (STNs) 349The first input vector, Qn, matches Zi exactly, so x\ begins to rise quickly.We shall assume that all other units remain quiescent. If Qn remains longenough, x\ will saturate. In any case, as soon as Qn is removed and Q i2 isapplied, x\ will begin to decay. At the same time, xi will begin to rise becauseQi2 matches the weight vector, 22. Moreover, since x\ will not have decayedaway completely, it will contribute to the positive input value to unit 2. WhenQi3 is applied, both x\ and XT_ will contribute to the input to unit 3.This cascade continues while all input vectors are applied to the network.Since the input vectors have been applied in the proper sequence, unit inputvalues tend to be reinforced by the outputs of preceding units. By the time thefinal input vector is applied, the network output, represented by y — x,,,, mayalready have reached saturation, even though contributions from units very earlyin the network may have decayed away.To illustrate the effects of a pattern mismatch, let's examine the situationwhere the patterns of Q\ are applied in reverse order. Since Qi,,, matches z,,,,x m will begin a quick rise toward saturation, although its output is not beingreinforced by any other units in the network. When Qi.,,,_i is applied, x n ,-\turns on and sends its output value to contribute to x m . The total input tothe mth unit is dx m -\, which, because d < 1, is unlikely to overcome thethreshold F. Therefore, x,,, will continue to decay away. By the time the lastinput vector is applied, x,,, may have decayed away entirely, indicating thatthe network has not recognized the STP. Note that we have assumed here thatQi.m-i is orthogonal to the weight vector z ni , and thus there is no contributionto the input to the mth unit from the dot product of these two vectors. Thisassumption is acceptable for this discussion to illustrate the concepts behind thenetwork operation. In practice, these vectors will not necessarily be orthogonal.Figure 9.7 shows a graphic example of unit outputs for a pattern recognitionby a simple, four-unit STN. Results from applying the identical input vectors,but in a random order, are shown in Figure 9.8. Notice how the activity patternappears to flow smoothly from left to right in the first figure.Because of the relatively rapid rise time/ of the unit activity, followed bya longer decay time, the STN has the property of being somewhat insensitiveto the speed at which the STP is presented. We know that different speakerspronounce the same word at slightly different rates; thus, tolerance of timevariation is a desirable characteristic of STNs. Figure 9.9 shows the results ofpresenting an STP at two different rates to the same network.Recall that the network we have been describing is capable of recognizingonly a single STP. To distinguish words in some specified vocabulary, we wouldhave to replicate the network for each word that we want the system to recognize.Figure 9.10 illustrates a system that can distinguish TV words.There are many aspects of the speech-recognition problem that we haveoverlooked in our discussion. For example, how do we account for both smallwords and large words? Moreover, some words are subsets of other words;will the system distinguish between a subset word spoken slowly and a superset