Deep-Learning-with-PyTorch

Down along the gradient
121
NOTE The normalization here absolutely helps get the network trained, but
you could make an argument that it’s not strictly needed to optimize the
parameters for this particular problem. That’s absolutely true! This problem is
small enough that there are numerous ways to beat the parameters into submission.
However, for larger, more sophisticated problems, normalization is an
easy and effective (if not crucial!) tool to use to improve model convergence.
Let’s run the loop for enough iterations to see the changes in params get small. We’ll
change n_epochs to 5,000:
# In[21]:
params = training_loop(
n_epochs = 5000,
learning_rate = 1e-2,
params = torch.tensor([1.0, 0.0]),
t_u = t_un,
t_c = t_c,
print_params = False)
params
# Out[21]:
Epoch 1, Loss 80.364342
Epoch 2, Loss 37.574917
Epoch 3, Loss 30.871077
...
Epoch 10, Loss 29.030487
Epoch 11, Loss 28.941875
...
Epoch 99, Loss 22.214186
Epoch 100, Loss 22.148710
...
Epoch 4000, Loss 2.927680
Epoch 5000, Loss 2.927648
tensor([ 5.3671, -17.3012])
Good: our loss decreases while we change parameters along the direction of gradient
descent. It doesn’t go exactly to zero; this could mean there aren’t enough iterations to
converge to zero, or that the data points don’t sit exactly on a line. As we anticipated, our
measurements were not perfectly accurate, or there was noise involved in the reading.
But look: the values for w and b look an awful lot like the numbers we need to use
to convert Celsius to Fahrenheit (after accounting for our earlier normalization when
we multiplied our inputs by 0.1). The exact values would be w=5.5556 and b=-
17.7778. Our fancy thermometer was showing temperatures in Fahrenheit the whole
time. No big discovery, except that our gradient descent optimization process works!