Rating Models and Validation - Oesterreichische Nationalbank

an indicator should only be used in cases where the empirical value of the indicator
in question agrees with the working hypothesis at least in the analysis sample.
In practice, working hypotheses are frequently tested in the analysis and validation
samples as well as the overall sample.
One alternative is to calculate the medians of each indicator separately for
the good and bad borrower groups. Given a sufficient quantity of data, it is also
possible to perform this calculation separately for all time periods observed.
This process involves reviewing whether the indicatorÕs median values differ significantly
for the good and bad groups of cases and correspond to the working
hypothesis (e.g. for G > B: the group median for good cases is greater than the
group median for bad cases). If this is not the case, the indicator is excluded
from further analyses.
Analyzing the IndicatorsÕ Availability and Dealing with Missing Values
The analysis of an indicatorÕs availability involves examining how often an
indicator cannot be calculated in relation to the overall sample of cases. We
can distinguish between two cases in which indicators cannot be calculated:
— The information necessary to calculate the indicator is not available in the
bank because it cannot be determined using the bankÕs operational processes
or IT applications. In such cases, it is necessary to check whether the use of
this indicator is relevant to credit ratings and whether it will be possible to
collect the necessary information in the future. If this is not the case, the
rating model cannot include the indicator in a meaningful way.
— The indicator cannot be calculated because the denominator is zero in a division
calculation. This does not occur very frequently in practice, as indicators
are preferably defined in such a way that this does not happen in meaningful
financial base values.
In multivariate analyses, however, a value must be available for each indicator
in each case to be processed, otherwise it is not possible to determine a rating
for the case. For this reason, it is necessary to handle missing values accordingly.
It is generally necessary to deal with missing values before an indicator is
transformed.
In the process of handling missing values, we can distinguish between four
possible approaches:
3. Cases in which an indicator cannot be calculated are excluded from the
development sample.
4. Indicators which do not attain a minimum level of availability are excluded
from further analyses.
5. Missing indicator values are included as a separate category in the analyses.
6. Missing values are replaced with estimated values specific to each group.
Procedure (1) is often impracticable because it excludes so many data records
from analysis that the data set may be rendered empirically invalid.
Procedure (2) is a proven method of dealing with indicators which are difficult
to calculate. The lower the fraction of valid values for an indicator in a sample,
the less suitable the indicator is for the development of a rating because its
value has to be estimated for a large number of cases. For this reason, it is necessary
to define a limit up to which an indicator is considered suitable for rating
development in terms of availability. If an indicator can be calculated in less than
approximately 80% of cases, it is not possible to ensure that missing values can
Rating Models and Validation
Guidelines on Credit Risk Management 77