Rating Models and Validation - Oesterreichische Nationalbank

Rating Models and Validation
aspects of the requirements imposed on banks using the IRB approach under
Basel II include: 75
— Design of the bankÕs internal processes which interface with the rating procedure
as well as their inclusion in organizational guidelines
— Use of the rating in risk management (in credit decision-making, risk-based
pricing, rating-based competence systems, rating-based limit systems, etc.)
— Conformity of the rating procedures with the bankÕs credit risk strategy
— Functional separation of responsibility for ratings from the front office
(except in retail business)
— Employee qualifications
— User acceptance of the procedure
— The userÕs ability to exercise freedom of interpretation in the rating procedure
(for this purpose, it is necessary to define suitable procedures and
process indicators such as the number of overrides)
Banks which intend to use an IRB approach will be required to document
these criteria completely and in a verifiable way. Regardless of this requirement,
however, complete and verifiable documentation of how rating models are used
should form an integral component of in-house use tests for any bank using a
rating system.
6.2 Quantitative Validation
In statistical models, quantitative validation represents a substantial part of
model development (cf. section 5.2). For heuristic and causal models, on the
other hand, an empirical data set is not yet available during rating development.
Therefore, the quantitative validation step is omitted during model development
in this family of models.
However, quantitative validation is required for all rating models. For this
purpose, validation should primarily use the data gained during practical operation
of the model. Comparison or benchmark data can also be included as a
supplement. This is particularly advisable when the performance of multiple
rating models is to be compared using a common sample.
The criteria to be reviewed in quantitative validation are as follows: 76
— Discriminatory power
— Calibration
— Stability
A sufficient data set for quantitative validation is available once all loans have
been rated for the first time (or re-rated) and observed over the forecasting
horizon of the rating model; this is usually the case approximately two years
after a new rating model is introduced.
6.2.1 Discriminatory Power
The term Òdiscriminatory powerÓ refers to the fundamental ability of a rating
model to differentiate between good and bad cases. 77 The term is often used
as a synonym for Òclassification accuracy.Ó In this context, the categories good
75 DEUTSCHE BUNDESBANK, Monthly Report for September 2003, Approaches to the validation of internal rating systems.
76 DEUTSCHE BUNDESBANK, Monthly Report for September 2003, Approaches to the validation of internal rating systems.
77 Instead of being restricted to borrower ratings, the descriptions below also apply to rating exposures in pools. For this reason, the
terms used are not differentiated.
98 Guidelines on Credit Risk Management