Deep-Learning-with-PyTorch

Good dogs vs. bad guys: False positives and false negatives
321
1. Guard dogs
2. Birds and
burglars
5. Balancing
POS
NEG
6. Augmentation
3. Ratios recaLl
and precision
4. new metric:
f1 score
7. Workin’ great!
Figure 12.3
The set of topics for this chapter, with a focus on the framing metaphor
Roxie will alert us to a burglar just about every time. She will also alert us to fire
engines, thunderstorms, helicopters, birds, the mail carrier, squirrels, passersby, and
so on. If we follow up on every bark, we’ll almost never get robbed (only the sneakiest
of sneak-thieves can slip past). Perfect! … Except that being that diligent means we
aren’t really saving any work by having a guard dog. Instead, we’ll be up every couple
of hours, flashlight in hand, due to Roxie having smelled a cat, or heard an owl, or
seen a late bus wander by. Roxie has a problematic number of false positives.
A false positive is an event that is classified as of interest or as a member of the
desired class (positive as in “Yes, that’s the type of thing I’m interested in knowing
about”) but that in truth is not really of interest. For the nodule-detection problem, it’s
when an actually uninteresting candidate is flagged as a nodule and, hence, in need of
a radiologist’s attention. For Roxie, these would be fire engines, thunderstorms, and
so on. We will use an image of a cat as the canonical false positive in the next section
and the figures that follow throughout the rest of the chapter.
Contrast false positives with true positives: items of interest that are classified correctly.
These will be represented in the figures by a human burglar.
Meanwhile, if Preston barks, call the police, since that means someone has almost
certainly broken in, the house is on fire, or Godzilla is attacking. Preston is a deep
sleeper, however, and the sound of an in-progress home invasion isn’t likely to rouse
him, so we’ll still get robbed just about every time someone tries. Again, while it’s better
than nothing, we’re not really ending up with the peace of mind that motivated us
to get a dog in the first place. Preston has a problematic number of false negatives.