Rating Models and Validation - Oesterreichische Nationalbank

4.1.5 Consistency
Heuristic models do not contradict recognized scientific theories and methods, as
these models are based on the experience and observations of credit experts.
In the data set used to develop empirical statistical rating models, relationships
between indicators may arise which contradict actual business considerations.
Such contradictory indicators have to be consistently excluded from further
analyses. Filtering out these problematic indicators will serve to ensure
consistency.
Causal models depict business interrelationships directly and are therefore
consistent with the underlying theory.
4.2 Suitability of Individual Model Types
The suitability of each model type is closely related to the data requirements for
the respective rating segments (see chapter 2). The most prominent question in
model evaluation is whether the quantitative and qualitative data used for credit
assessment in individual segments can be processed properly. While quantitative
data generally fulfills this condition in all models, differences arise with regard
to qualitative data in statistical models.
In terms of discriminatory power and calibration, statistical models demonstrate
clearly superior performance in practice compared to heuristic models.
Therefore, banks are increasingly replacing or supplementing heuristic models
with statistical models in practice. This is especially true in those segments for
which it is possible to compile a sufficient data set for statistical model development
(in particular corporate customers and mass-market banking). For these
customer segments, statistical models are the standard.
However, the quality and suitability of the rating model used cannot be
assessed on the basis of the model type alone. Rather, validation should involve
regular reviews of a rating modelÕs quality on the basis of ongoing operations.
Therefore, we only describe the essential, observable strengths and weaknesses
of the rating models for each rating segment below, without attempting to recommend,
prescribe or rule out rating models in individual segments.
4.2.1 Heuristic Models
In principle, heuristic models can be used in all rating segments. However, in
terms of discriminatory power, statistical models are clearly superior to heuristic
models in the corporate customer segment and in mass-market banking.
Therefore, the use of statistical models is preferable in those particular segments
if a sufficient data set is available.
When heuristic models are used in practice, it is important in any case to
review their discriminatory power and forecasting accuracy in the course of validation.
Classic Rating Questionnaires
The decisive success component in a classic rating questionnaire is the use of
creditworthiness criteria for which the user can give clear and understandable
answers. This will increase user acceptance as well as the objectivity of the
model. Another criterion is the plausible and comprehensible assignment of
points to specific answers. Answers which experience has shown to indicate high
Rating Models and Validation
Guidelines on Credit Risk Management 57