Deep-Learning-with-PyTorch

416 CHAPTER 14 End-to-end nodule analysis, and where to go next
This candidate is assigned a 53% probability of being malignant, so it barely makes the probability
threshold of 50%. The malignancy classification assigns a very low (3%) probability.
nodule prob 0.533, malignancy prob 0.030, center xyz XyzTuple
➥ (x=-128.857421875, y=-80.349609375, z=-31.300007820129395)
nodule prob 0.754, malignancy prob 0.446, center xyz XyzTuple
➥ (x=-116.396484375, y=-168.142578125, z=-238.30000233650208)
...
nodule prob 0.974, malignancy prob 0.427, center xyz XyzTuple
➥ (x=121.494140625, y=-45.798828125, z=-211.3000030517578)
nodule prob 0.700, malignancy prob 0.310, center xyz XyzTuple
➥ (x=123.759765625, y=-44.666015625, z=-211.3000030517578)
...
Detected as a nodule with very high confidence
and assigned a 42% probability of malignancy
Scan ID
The script found 16 nodule candidates in total. Since we’re using our validation set, we
have a full set of annotations and malignancy information for each CT, which we can
use to create a confusion matrix with our results. The rows are the truth (as defined by
the annotations), and the columns show how our project handled each case:
1.3.6.1.4.1.14519.5.2.1.6279.6001.592821488053137951302246128864
| Complete Miss | Filtered Out | Pred. Nodule
Non-Nodules | | 1088 | 15
Benign | 1 | 0 | 0
Malignant | 0 | 0 | 1
The rows contain the ground truth.
The Complete Miss column is when our segmenter did not flag a nodule at all. Since
the segmenter was not trying to flag non-nodules, we leave that cell blank. Our segmenter
was trained to have high recall, so there are a large number of non-nodules,
but our nodule classifier is well equipped to screen those out.
So we found the 1 malignant nodule in this scan, but missed a 17th benign one. In
addition, 15 false positive non-nodules made it through the nodule classifier. The
filtering by the classifier brought the false positives down from over 1,000! As we saw
earlier, 1,088 is about O(2 10 ), so that lines up with what we expect. Similarly, 15 is
about O(2 4 ), which isn’t far from the O(2 5 ) we ballparked.
Cool! But what’s the larger picture?
14.4 Quantitative validation
Prognosis: Complete Miss means the segmentation
didn’t find a nodule, Filtered Out is the classifier’s work,
and Predicted Nodules are those it marked as nodules.
Now that we have anecdotal evidence that the thing we built might be working on one
case, let’s take a look at the performance of our model on the entire validation set.
Doing so is simple: we run our validation set through the previous prediction and
check how many nodules we get, how many we miss, and how many candidates are
erroneously identified as nodules.