Rating Models and Validation - Oesterreichische Nationalbank

Rating Models and Validation
available, the binomial test can be replaced with a test using standard normal
distribution. 109
Chart 76 below shows an example of how the binomial test would be performed
for one row in a transition matrix. For each matrix field, we first calculate
the test value on the left side of the inequality. We then compare this test
value to the right side of the inequality for the 90% and 95% significance levels.
Significant deviations between the forecast and realized transition rates are identified
at the 90% level for transitions to classes 1b, 3a, 3b, and 3e. At the 95%
significance level, only the underruns of forecast rates of transition to classes 1b
and 3b are significant.
In this context, class 1a is a special case because the forecast transition rate
equals zero in this class. In this case, the test value used in the binomial test
always equals 1, even if no transitions are observed. However, this is not considered
a misestimate of the transition rate. If transitions are observed, the transition
rate is obviously not equal to zero and would have to be adjusted accordingly.
Chart 76: Data Example for Back-testing a Transition Matrix with the Binomial Test
If significant deviations are identified between the forecast and realized transition
rates, it is necessary to adjust the transition matrix. In the simplest of
cases, the inaccurate values in the matrix are replaced with new values which
are determined empirically. However, this can also bring about inconsistencies
in the transition matrix which then make it necessary to smooth the matrix.
There is no simple algorithmic method which enables the parallel adjustment
of all transition frequencies to the required significance levels with due
attention to the consistency rules. Therefore, practitioners often use pragmatic
solutions in which parts of individual transition probabilities are shifted to
neighboring classes out of heuristic considerations in order to adhere to the consistency
rules. 110
If multi-year transition matrices can be calculated directly from the data set,
it is not absolutely necessary to adhere to the consistency rules. However, what
is essential to the validity and practical viability of a rating model is the consis-
109 Cf. the comments in section 6.2.2 on reviewing the significance of default rates.
110 In mathematical terms, it is not even possible to ensure the existence of a solution in all cases.
126 Guidelines on Credit Risk Management