The Effects of Sanction Intensity on Criminal Conduct - JDAI Helpdesk

appears to be a good classification threshold.
Here, the model’s sensitivity and
specificity are most balanced compared to other cut-<strong>of</strong>f points (Sn = 63.0%; Sp = 65.8%).
This means that the probability that a non-serious <strong>of</strong>fender received a low risk prediction
and the probability that a serious <strong>of</strong>fender received a non-low risk prediction are roughly
the same. Of the <strong>of</strong>fenders receiving a low-risk prediction, 96.6 per cent were in fact
low-risk.
<strong>The</strong> ‘worst case scenario’ false positive rate (low-risk <strong>of</strong>fenders who
committed serious <strong>of</strong>fenses) is very low, at 3.4 per cent.
Table 3.5 suggests that the cut-<strong>of</strong>f point should not be set below 0.5. Although
the positive predictive value remains high at thresholds <strong>of</strong> 0.45 and below, there is a
considerable loss <strong>of</strong> specificity at the expense <strong>of</strong> the less important sensitivity (Sn =
71.3%; Sp = 55.1% at 0.45 threshold). <strong>The</strong> probability <strong>of</strong> finding a false positive also
begins to increase. On the other hand, there may be a case for increasing the cut-<strong>of</strong>f point
to 0.55, but no more. At 0.55, the positive predictive value increases to 97.3 per cent,
specificity increases to 78.5 per cent, and the likelihood <strong>of</strong> a false positive drops to 2.7
per cent. However, the sensitivity drops to just over 50 per cent, which starts to raise
questions about the model’s ability to meet its purpose. Thus, a threshold between 0.5
and 0.55 appears to provide the best trade-<strong>of</strong>f between all the factors discussed above.
Tables 3.6 and 3.7 show the same analyses repeated for UCR Part I and
victim/damage <strong>of</strong>fenses. Note that the model is not designed to predict these <strong>of</strong>fense
types (as the slightly lower positive predictive values in these two tables suggest), so the
results from Table 3.5 should be taken as the definitive examination <strong>of</strong> the threshold.
However, we use these additional outcomes to examine whether the cut-<strong>of</strong>f point allows
too many <strong>of</strong>fenders who might be considered ‘serious’ by alternative standards to be
159