Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

Define - Model Training V2
1 %%writefile model_training/v2.py
2
3 # Defines number of epochs
4 n_epochs = 1000
5
6 losses = []
7
8 # For each epoch...
9 for epoch in range(n_epochs):
10 # inner loop
11 mini_batch_losses = [] 4
12 for x_batch, y_batch in train_loader: 1
13 # the dataset "lives" in the CPU, so do our mini-batches
14 # therefore, we need to send those mini-batches to the
15 # device where the model "lives"
16 x_batch = x_batch.to(device) 2
17 y_batch = y_batch.to(device) 2
18
19 # Performs one train step and returns the
20 # corresponding loss for this mini-batch
21 mini_batch_loss = train_step_fn(x_batch, y_batch) 3
22 mini_batch_losses.append(mini_batch_loss) 4
23
24 # Computes average loss over all mini-batches
25 # That's the epoch loss
26 loss = np.mean(mini_batch_losses) 5
27
28 losses.append(loss)
1 Mini-batch inner loop
2 Sending one mini-batch to the device
3 Performing a training step
4 Keeping track of the loss inside each mini-batch
5 Averaging losses of mini-batches to get epoch’s loss
140 | Chapter 2: Rethinking the Training Loop