Rating Models and Validation - Oesterreichische Nationalbank

Rating Models and Validation
For the purpose of validating rating models, the condition in the definition of
conditional entropy is the classification in rating class c and default event to be
depicted ðDÞ. For each rating class c, the conditional entropy is hc:
hc ¼ fpðDjcÞ log 2 ðpðDjcÞÞ þ ð1 pðDjcÞÞ log 2 ð1 pðDjcÞÞg:
The conditional entropy hc of a rating class thus corresponds to the uncertainty
remaining with regard to the future default status after a case is assigned
to a rating class. Across all rating classes in a model, the conditional entropy H1
(averaged using the observed frequencies of the individual rating classesÕ pc) is
defined as:
H1 ¼ X
c
pc hc:
The average conditional entropy H1 corresponds to the uncertainty remaining
with regard to the future default status after application of the rating model.
Using the entropy H0, which is available without applying the rating model if
the average default probability of the sample is known, it is possible to define
a relative measure of the information gained due to the rating model. The conditional
information entropy ratio (CIER) is defined as: 93
CIER ¼ H0 H1
¼ 1 H1
:
H0
The value CIER can be interpreted as follows:
— If no additional information is gained by applying the rating model, H1 ¼ H0
and CIER ¼ 0.
— If the rating model is ideal and no uncertainty remains regarding the default
status after the model is applied, H1 ¼ 0 and CIER ¼ 1.
The higher the CIER value is, the more information regarding the future
default status is gained from the rating system.
However, it should be noted that information on the properties of the rating
model is lost in the calculation of CIER, as is the case with AUC and the other
one-dimensional measures of discriminatory power. As an individual indicator,
therefore, CIER has only limited meaning in the assessment of a rating model.
Chart 66: Entropy-Based Measures of Discriminatory Power for the Data Example
93 The difference ðH0 H1Þ is also referred to as the Kullback-Leibler distance. Therefore, CIER is a standardized Kullback-
Leibler distance.
114 Guidelines on Credit Risk Management
H0