Deep-Learning-with-PyTorch

Summary
139
5.6 Conclusion
We started this chapter with a big question: how is it that a machine can learn from
examples? We spent the rest of the chapter describing the mechanism with which a
model can be optimized to fit data. We chose to stick with a simple model in order to
see all the moving parts without unneeded complications.
Now that we’ve had our fill of appetizers, in chapter 6 we’ll finally get to the main
course: using a neural network to fit our data. We’ll work on solving the same
thermometer problem, but with the more powerful tools provided by the torch.nn
module. We’ll adopt the same spirit of using this small problem to illustrate the
larger uses of PyTorch. The problem doesn’t need a neural network to reach a
solution, but it will allow us to develop a simpler understanding of what’s required to
train a neural network.
5.7 Exercise
5.8 Summary
1 Redefine the model to be w2 * t_u ** 2 + w1 * t_u + b.
a What parts of the training loop, and so on, need to change to accommodate
this redefinition?
b What parts are agnostic to swapping out the model?
c Is the resulting loss higher or lower after training?
d Is the actual result better or worse?
• Linear models are the simplest reasonable model to use to fit data.
• Convex optimization techniques can be used for linear models, but they do not
generalize to neural networks, so we focus on stochastic gradient descent for
parameter estimation.
• Deep learning can be used for generic models that are not engineered for solving
a specific task, but instead can be automatically adapted to specialize themselves
on the problem at hand.
• Learning algorithms amount to optimizing parameters of models based on
observations. A loss function is a measure of the error in carrying out a task,
such as the error between predicted outputs and measured values. The goal is
to get the loss function as low as possible.
• The rate of change of the loss function with respect to the model parameters
can be used to update the same parameters in the direction of decreasing loss.
• The optim module in PyTorch provides a collection of ready-to-use optimizers
for updating parameters and minimizing loss functions.
• Optimizers use the autograd feature of PyTorch to compute the gradient for
each parameter, depending on how that parameter contributes to the final output.
This allows users to rely on the dynamic computation graph during complex
forward passes.