Deep-Learning-with-PyTorch

346 CHAPTER 12 Improving training with metrics and augmentation
Overfitting is usually due to having too few training samples when compared to the
ability of the model to just memorize the answers. The median human can memorize
the birthdays of their immediate family but would have to resort to generalizations
when predicting the ages of any group larger than a small village.
Our face-to-age model has the capacity to simply memorize the photos of anyone
who doesn’t look exactly their age. As we discussed in part 1, model capacity is a somewhat
abstract concept, but is roughly a function of the number of parameters of the
model times how efficiently those parameters are used. When a model has a high
capacity relative to the amount of data needed to memorize the hard samples from
the training set, it’s likely that the model will begin to overfit on those more difficult
training samples.
12.6 Preventing overfitting with data augmentation
It’s time to take our model training from good to great. We need to cover one last step
in figure 12.20.
1. Guard dogs
2. Birds and
burglars
5. Balancing
POS
NEG
6.
Augmentation
3. Ratios recaLl
and precision
4. new metric:
f1 score
7. Workin’ great!
Figure 12.20
The set of topics for this chapter, with a focus on data augmentation
We augment a dataset by applying synthetic alterations to individual samples, resulting
in a new dataset with an effective size that is larger than the original. The typical goal
is for the alterations to result in a synthetic sample that remains representative of the
same general class as the source sample, but that cannot be trivially memorized alongside
the original. When done properly, this augmentation can increase the training set