Deep-Learning-with-PyTorch

144 CHAPTER 6 Using a neural network to fit the data
can be a single scalar or matrix, while b is a scalar or vector (the dimensionality of the
inputs and weights must match, however). In the latter case, the previous expression is
referred to as a layer of neurons, since it represents many neurons via the multidimensional
weights and biases.
6.1.1 Composing a multilayer network
A multilayer neural network, as represented in figure 6.3, is made up of a composition
of functions like those we just discussed
x_1 = f(w_0 * x + b_0)
x_2 = f(w_1 * x_1 + b_1)
...
y = f(w_n * x_n + b_n)
where the output of a layer of neurons is used as an input for the following layer.
Remember that w_0 here is a matrix, and x is a vector! Using a vector allows w_0 to
hold an entire layer of neurons, not just a single weight.
0
OUTPUT
A NEURAL NETWORK
INPUT
= 2 1 X + 1 + 2 +
NEURON
LEARNED PARAMETERS
X
INPUT
LEARNED
LAYER
ACTIVATION
LAYER
LAYER
0
OUTPUT
Figure 6.3 A neural network
with three layers
6.1.2 Understanding the error function
An important difference between our earlier linear model and what we’ll actually be
using for deep learning is the shape of the error function. Our linear model and
error-squared loss function had a convex error curve with a singular, clearly defined
minimum. If we were to use other methods, we could solve for the parameters minimizing
the error function automatically and definitively. That means that our parameter
updates were attempting to estimate that singular correct answer as best they could.