Rating Models and Validation - Oesterreichische Nationalbank

The table below (chart 67) shows a comparison of Gini Coefficient values
and CIER discriminatory power indicators from a study of rating models for
American corporates.
Chart 67: Gini Coefficient and CIER Values from a Study of American Corporates 94
6.2.2 Back-Testing the Calibration
The assignment of default probabilities to a rating modelÕs output is referred to
as calibration. The quality of calibration depends on the degree to which the
default probabilities predicted by the rating model match the default rates
actually realized. Therefore, reviewing the calibration of a rating model is frequently
referred to as back-testing.
The basic data used for back-testing are: the default probabilities forecast
over a rating class for a specific time horizon (usually 12 months), the number
of cases assigned to the respective rating class by the model, and the default status
of those cases once the forecasting period has elapsed, starting from the time
of rating (i.e. usually 12 months after the rating was assigned). Calibration
involves assigning forecast default probabilities to the individual rating classes
(cf. section 5.3). In this process, it is also possible to use longer forecasting horizons
than the 12-month horizon required of IRB banks; these other time horizons
also have to undergo back-testing.
The results of various segment-specific rating procedures are frequently
depicted on a uniform master scale of default probabilities.
In the course of quantitative validation, significant differences may be identified
between the default rates on the master scale and the default rates actually
realized for individual rating classes in a segment-specific rating procedure. In
order to correct these deviations, two different approaches are possible:
— In a fixed master scale, the predefined default probabilities are not changed;
instead, only the assignment of results from the rating procedure under
review to rating classes on the master scale is adjusted.
— In a variable master scale, the predefined default probabilities are changed,
but the assignment of rating results from the rating procedure under review
to rating classes on the master scale is not adjusted.
As any changes to the master scale will affect all of the rating procedures
used in a bank — including those for which no (or only minor) errors in calibration
have been identified — fixed master scales are generally preferable. This is
especially true in cases where the default probability of rating classes serves as
the basis for risk-based pricing, which would be subject to frequent changes if a
variable master scale were used.
94 Cf. KEENAN/SOBEHART, Performance Measures.
Rating Models and Validation
Guidelines on Credit Risk Management 115