Rating Models and Validation - Oesterreichische Nationalbank

Rating Models and Validation
Chart 72: Theoretical Minimum Number of Cases for the Normal Test based on Various Default Rates
The table below (chart 73) shows the results of the binomial test for significant
underestimates of the default rate. It is indeed conspicuous that the results
largely match those of the test using normal distribution despite the fact that
several classes did not contain the required minimum number of cases.
Chart 73: Identification of Significant Deviations in Calibration for the Data Example (Binomial Test)
One interesting deviation appears in rating class 1, where the binomial test
yields a significant underestimate at the 95% level for an estimated default probability
of 0.05% and no observed defaults. In this case, the binomial test does
not yield reliable results, as the following already applies when no defaults are
observed:
P n 0jp prognose
1 ; N1 ¼ð1 p prognose
1 Þ N1 > 90%:
In general, the test using normal distribution is faster and easier to perform,
and it yields useful results even for small samples and low default rates. This test
is thus preferable to the binomial test even if the mathematical prerequisites for
its application are not always met. However, it is important to bear in mind that
the binomial test and its generalization using normal distribution are based on
the assumption of uncorrelated defaults. A test procedure which takes default
correlations into account is presented below.
Calibration Test Procedure Based on Default Correlation
The assumption of uncorrelated defaults generally yields an overestimate of the
significance of deviations in the realized default rate from the forecast rate. This
is especially true of risk underestimates, that is, cases in which the realized
default rate is higher than the forecast rate. From a conservative risk assessment
standpoint, overestimating significance is not critical in the case of risk under-
122 Guidelines on Credit Risk Management