Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub

torch.manual_seed(44)
dropping_model.train()
output_train = dropping_model(spaced_points)
output_train
Output
tensor([0.0000, 0.4000, 0.0000, 0.8000, 0.0000, 1.2000, 1.4000,
1.6000, 1.8000, 0.0000, 2.2000])
There are many things to notice here:
• The model is in train mode (very important, hold on to this!).
• Since this model does not have any weights, it becomes clear that dropout
drops inputs, not weights.
• It dropped four elements only!
• The remaining elements have different values now!
"What’s going on here?"
First, dropping is probabilistic, so each input had a 50% chance of being dropped.
In our tiny example, by chance, only four out of ten were actually dropped (hold on
to this thought too!).
Figure 6.6 - Applying dropout
Second, the remaining elements need to be proportionally adjusted by a factor of
1/p. In our example, that’s a factor of two.
Dropout | 433