POLITECHNIKA WARSZAWSKA

3. Basics of Artificial Neural Network (ANN)
where elements w j , are called synapse weights, can be modified during the learning
process.
The output of the neuron unit is defined as follows:
y = F(e) (3.3)
Note, that w 0 - is adjustable bias and F – is activation function (also called transfer
function). Thus, the output, y, is obtained by summing the weighted inputs and
passing the results through a nonlinear (or linear) activation function F.
The activation function F map, a weighted sum's e (possibly) infinite domain
to a prespecified range. Although the number of F functions is possibly infinite, six
types are regularly applied in the majority of ANN: linear, step, bipolar, sigmoid,
hyperbolic tangent. With the exception of the linear F function, all of these functions
introduce a nonlinearity in the network by bounding the output within a fixed range.
In the next subsection some examples of commonly used activation functions are
briefly presented.
3.2.1. Activation functions
The linear F function (Fig. 3.2) produces a linearly modulated output from the input e
as described by equation
F(e) = ξ e (3.3)
F (e)
F (e)
1
1
0
e
0
e
-1
-1
-1 0 1
-1 0 1
Fig. 3.2. Linear activation function
16