Modeling Hydra Behavior Using Methods Founded in Behavior-Based Robotics

Appendix B. Recurrent neural networks 59case with FFNN). Instead, the network simply consists of a set of neurons with possibleconnections between all neurons, as well as self-couplings. An example of an RNN isshown in Fig. B.2. For neurons connected in a network, the weighted incoming connections(denoted x 1 , ..., x k in Fig. B.1) consists of both external inputs, I, and output signalsfrom neurons in the network, y). The dynamics of neuron i in a network consisting of nneurons and m input signals, is governed by Eq. B.1.(τ idy idt + y i = σb i +n∑w ij y j +j=1)m∑wijI I j , i = 1, ..., n, (B.1)where τ i is a time constant, b i the bias term, y i is the output of neuron i, w ij the synapticweight connecting the output of neuron j to neuron i, w I ij the weight connecting inputsignal j to neuron i, and I j is the external input from input signal j to neuron i. Severalalternative activation functions, σ(·), are common. In this project, it was taken as1σ(z) = , (B.2)1 + e−cz which restricts the output of any neuron to the range [0, 1]. For numerical (computer)calculations, the model is discretized, using Euler’s method, according tody idt ≈ y i(t + ∆t) − y i (t). (B.3)∆tUsing Eq. B.3 in Eq. B.1 gives()y i (t + ∆t) − y i (t)n∑m∑τ i + y i (t) = σ b i + w ij y j (t) + w I∆tijI j (t) , (B.4)so the discrete model can be expressed as:[ (y i (t + ∆t) = y i (t) + ∆t −y i (t) + σ b i +τ iB.2 Learning algorithmsj=1j=1n∑w ij y j (t) +j=1j=1)]m∑wijI I j (t) . (B.5)The parameters of an ANN is assigned in a process known as training, where a learningalgorithm is applied in order to optimize the network parameters, i.e. the synapticweights, bias terms, and time constants, in the case of RNNs. Some learning algorithmsalso allows the size and structure of the network to be optimized as well. For FFNNs,backpropagation is a commonly used learning algorithm, see e.g. [23, 67]. RNNs are, ingeneral, more complex to train than FFNNs, but one way of accomplishing parametricaland structural optimization of RNNs is to use EAs, which was also done in this project.j=1