Rating Models and Validation - Oesterreichische Nationalbank

Rating Models and Validation
Chart 69: Calculation of Brier Score for the Data Example
In practice, a standardized measure known as the Brier Skill Score (BSS) is
often used instead of the Brier Score. This measure scales the Brier Score to the
variance term:
BSS ¼ 1
BS
p observed ð1 p observed Þ :
In the trivial, ideal model, the Brier Skill Score takes the value zero. For the
example above, the resulting value is BSS = 4.04 %.
Reliability Diagrams
Additional information on the quality of calibration for rating models can be
derived from the reliability diagram, in which the observed default rates are
plotted against the forecast rate in each rating class. The resulting curve is often
referred to as the calibration curve.
Chart 70: Reliability Diagram for the Data Example
118 Guidelines on Credit Risk Management