Rating Models and Validation - Oesterreichische Nationalbank

Chart 71: Identification of Significant Deviations in Calibration in the Data Example
(Test with Standard Normal Distribution)
For the purpose of interpreting confidence levels, a Òtraffic lights approachÓ
has been proposed for practice in Germany. 101 In this approach, deviations of
realized and forecast default rates below a confidence level of 95% should
not be regarded as significant (ÒgreenÓ range). Deviations at a confidence level
of at least 99.9% are then considered significant and should definitely be corrected
(ÒredÓ range). Deviations which are significant at confidence levels
between 95% and 99.9% may need to be corrected (ÒyellowÓ range).
For the example above, this means that the overall default rate — which was
underestimated by the rating model — should be corrected upward, preferably
by making the appropriate adjustments in rating classes 5 to 7.
Binomial Calibration Test
The test using normal distribution (described above) is a generalization of the
binomial test for frequencies of uncorrelated binary events. The binomial test
is described in detail below.
For low default probabilities and low numbers of cases in the individual
rating classes, the prerequisites for using normal distribution are not always
met. The table below (chart 72) lists the minimum number of cases required
for a sound approximation of test values with standard normal distribution
when testing various default probabilities. 102
If individual classes contain fewer cases than the minimum number indicated,
the binomial test should be carried out. In the formula below, N c
denotes the number of defaults observed in class c, and Nc refers to the number
of cases in class c. Summation is performed for all defaults in class c.
— (one-sided test): If
X N c
n¼0
Nc
n ðpforecast
c Þ n ð1 p forecast Þ Nc n >q;
the default rate in class c is significantly underestimated at confidence level
q. 103
101 Cf. TASCHE, D., A traffic lights approach to PD validation.
102 The strict condition for the application of standard normal distribution as an approximation of binomial distribution is
Npð1 pÞ > 9, where N is the number of cases in the rating class examined and p is the forecast default probability (cf.
SACHS, L., Angewandte Statistik, p. 283).
103 This is equivalent to the statement that the probability of occurrence P ½n