Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Let’s take a closer look at its components:
• blue boxes ((1)s): these boxes correspond to the tensors we use as
parameters, the ones we’re asking PyTorch to compute gradients for
• gray boxes (MulBackward0 and AddBackward0): Python operations that involve
gradient-computing tensors or its dependencies
• green box ((80, 1)): the tensor used as the starting point for the computation
of gradients (assuming the backward() method is called from the variable used
to visualize the graph)—they are computed from the bottom-up in a graph
Now, take a closer look at the gray box at the bottom of the graph: Two arrows are
pointing to it since it is adding up two variables, b and w*x. Seems obvious, right?
Then, look at the other gray box (MulBackward0) of the same graph: It is performing
a multiplication operation, namely, w*x. But there is only one arrow pointing to it!
The arrow comes from the blue box that corresponds to our parameter w.
"Why don’t we have a box for our data (x)?"
The answer is: We do not compute gradients for it!
So, even though there are more tensors involved in the operations performed by
the computation graph, it only shows gradient-computing tensors and their
dependencies.
What would happen to the computation graph if we set requires_grad to False for
our parameter b?
b_nograd = torch.randn(1, requires_grad=False, \
dtype=torch.float, device=device)
w = torch.randn(1, requires_grad=True, \
dtype=torch.float, device=device)
yhat = b_nograd + w * x_train_tensor
make_dot(yhat)
94 | Chapter 1: A Simple Regression Problem