Neural Networks - Algorithms, Applications,and ... - Csbdu.in

20 Introduction to ANS Technologyconverts the net input value, net;, to the node's output value, Xj. In this text, weshall consistently use the term output function for /,() of Eqs. (1.3) and (1.4).Be aware, however, that the literature is not always consistent in this respect.When we are describing the mathematical basis for network models, it willoften be useful to think of the network as a dynamical system—that is, asa system that evolves over time. To describe such a network, we shall writedifferential equations that describe the time rate of change of the outputs of thevarious PEs. For example, ±, — gi(x t , net,) represents a general differentialequation for the output of the ith PE, where the dot above the x refers todifferentiation with respect to time. Since netj depends on the outputs of manyother units, we actually have a system of coupled differential equations.As an example, let's look at the equation±i = -Xi + /j(neti)for the output of the itii processing element. We apply some input values to thePE so that net; > 0. If the inputs remain for a sufficiently long time, the outputvalue will reach an equilibrium value, when x, = 0, given bywhich is identical to Eq. (1.4). We can often assume that input values remainuntil equilibrium has been achieved.Once the unit has a nonzero output value, removal of the inputs will causethe output to return to zero. If net; = 0, thenwhich means that x —> 0.It is also useful to view the collection of weight values as a dynamicalsystem. Recall the discussion in the previous section, where we asserted thatlearning is a result of the modification of the strength of synaptic junctions betweenneurons. In an ANS, learning usually is accomplished by modification ofthe weight values. We can write a system of differential equations for the weightvalues, Wij = G Z (WJJ, z;,Xj,...), where G, represents the learning law. Thelearning process consists of finding weights that encode the knowledge that wewant the system to learn. For most realistic systems, it is not easy to determinea closed-form solution for this system of equations. Techniques exist, however,that result in an acceptable approximation to a solution. Proving the existenceof stable solutions to such systems of equations is an active area of research inneural networks today, and probably will continue to be so for some time.1.2.2 Vector FormulationIn many of the network models that we shall discuss, it is useful to describecertain quantities in terms of vectors. Think of a neural network composed ofseveral layers of identical processing elements. If a particular layer contains n