Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

This is the general structure you’ll use over and over again for
training PyTorch models.
Sure, a different dataset and problem will require a different
model and loss function, and you may want to try a different
optimizer and a cycling learning rate (we’ll get to that later!), but
the rest is likely to remain exactly the same.
Recap
We have covered a lot of ground in this chapter:
• writing a higher-order function that builds functions to perform training steps
• understanding PyTorch’s Dataset and TensorDataset classes, implementing its
__init__(), __getitem__(), and __len__() methods
• using PyTorch’s DataLoader class to generate mini-batches out of a dataset
• modifying our training loop to incorporate mini-batch gradient descent logic
• writing a helper function to handle the mini-batch inner loop
• using PyTorch’s random_split() method to generate training and validation
datasets
• writing a higher-order function that builds functions to perform validation
steps
• realizing the importance of including model.eval() inside the validation loop
• remembering the purpose of no_grad() and using it to prevent any kind of
gradient computation during validation
• using SummaryWriter to interface with TensorBoard for logging
• adding a graph representing our model to TensorBoard
• sending scalars to TensorBoard to track the evolution of training and validation
losses
• saving / checkpointing and loading models to and from disk to allow resuming
model training or deployment
• realizing the importance of setting the mode of the model: train() or eval(),
for checkpointing or deploying for prediction, respectively
Recap | 173