Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

notebook. If not, just click on Chapter00.ipynb on your Jupyter’s home page.
Imports
For the sake of organization, all libraries needed throughout the code used in any
given chapter are imported at its very beginning. For this chapter, we’ll need the
following imports:
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
Visualizing Gradient Descent
According to Wikipedia [35] : "Gradient descent is a first-order
iterative optimization algorithm for finding a local minimum of a
differentiable function."
But I would go with: "Gradient descent is an iterative technique
commonly used in machine learning and deep learning to find the
best possible set of parameters / coefficients for a given model,
data points, and loss function, starting from an initial, and usually
random, guess."
"Why visualizing gradient descent?"
I believe the way gradient descent is usually explained lacks intuition. Students and
beginners are left with a bunch of equations and rules of thumb—this is not the way
one should learn such a fundamental topic.
If you really understand how gradient descent works, you will also understand how
the characteristics of your data and your choice of hyper-parameters (mini-batch
size and learning rate, for instance) have an impact on how well and how fast the
model training is going to be.
By really understanding, I do not mean working through the equations manually: this
does not develop intuition either. I mean visualizing the effects of different
settings; I mean telling a story to illustrate the concept. That’s how you develop
intuition.
Visualizing Gradient Descent | 23