Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Chapter 0
Visualizing Gradient Descent
Spoilers
In this chapter, we will:
• define a simple linear regression model
• walk through every step of gradient descent: initializing parameters,
performing a forward pass, computing errors and loss, computing gradients,
and updating parameters
• understand gradients using equations, code, and geometry
• understand the difference between batch, mini-batch, and stochastic gradient
descent
• visualize the effects on the loss of using different learning rates
• understand the importance of standardizing / scaling features
• and much, much more!
There is no actual PyTorch code in this chapter… it is Numpy all along because our
focus here is to understand, inside and out, how gradient descent works. PyTorch
will be introduced in the next chapter.
Jupyter Notebook
The Jupyter notebook corresponding to Chapter 0 [33] is part of the official Deep
Learning with PyTorch Step-by-Step repository on GitHub. You can also run it
directly in Google Colab [34] .
If you’re using a local installation, open your terminal or Anaconda prompt and
navigate to the PyTorchStepByStep folder you cloned from GitHub. Then, activate
the pytorchbook environment and run jupyter notebook:
$ conda activate pytorchbook
(pytorchbook)$ jupyter notebook
If you’re using Jupyter’s default settings, this link should open Chapter 0’s
22 | Chapter 0: Visualizing Gradient Descent