Rating Models and Validation - Oesterreichische Nationalbank

Rating Models and Validation
optimize the model should be made in cases where the difference in discriminatory
power between the analysis and validation samples exceeds 10% as measured
in Powerstat values.
Moreover, the stability of discriminatory power in the analysis and validation
samples also has to be determined for time periods other than the forecasting
horizon used to develop the model. Suitable scoring functions should show
sound discriminatory power for forecasting horizons of 12 months as well as
longer periods.
Significance of Individual Coefficients
In the optimization of indicator coefficients, a statistical hypothesis in the form
of Òcoefficient ? 0Ó is postulated. This hypothesis can be tested using the significance
measures (e.g. F-Tests) produced by most optimization programs. 59 On
the basis of this information, it is also possible to realize algorithms for automatic
indicator selection. In this context, all indicators whose optimized coefficients
are not equal to zero at a predefined level of significance are selected
from the sample. These algorithms are generally included in software packages
for multivariate analysis. 60
Coverage of Relevant Information Categories
An important additional requirement condition in the development of scoring
functions is the coverage of all information categories (where possible). This
ensures that the rating represents a holistic assessment of the borrowerÕs economic
situation.
Should multivariate analysis yield multiple scoring functions which are
equivalent in terms of the criteria described, the scoring function which contains
the most easily understandable indicators should be chosen. This will also
serve to increase user acceptance.
Once the scoring function has been selected, it is possible to scale the score
values (e.g. to a range of 0 to 100). This enables partial scores from various
information categories to be presented in a simpler and more understandable
manner.
5.2.3 Overall Scoring Function
If separate partial scoring functions are developed for quantitative and qualitative
data, these functions have to be linked in the modelÕs architecture to form
an overall scoring function. The objective in this context is to determine the
optimum weighting of the two data types.
In general, the personal traits of the business owner or manager influence
the credit quality of enterprises in smaller-scale rating segments more heavily
than in larger companies. For this reason, we can observe in practice that the
influence of qualitative information categories on each overall scoring function
increases as the size of the enterprises in the segment decreases. However, the
weighting shown in chart 37 is only to be seen as a rough guideline and not as a
binding requirement of all rating models suitable for use in practice.
59 Cf. (for example) SACHS, L., Angewandte Statistik, section 3.5.
60 e.g. SPSS.
82 Guidelines on Credit Risk Management