The tenth IMSC, Beijing, China, 2007 - International Meetings on ...

chosen since it completely defines the variability of cases and hence the verification score.
<strong>The</strong> trouble is that weather is highly correlated in time and space thus having much less
variability then the independent normative model indicates. <strong>The</strong> effect of correlation can be
incorporated by using randomly SIMULATED VERIFICATION: generated forecasts and
observation with known correlation.
SYNTHETIC VERIFICATION is the closed form result which would occur from a series of
simulated verifications and thus an explicit form shows the effects of skill, event climate
probability and other factors on a verification score and one easily programmed simulation of
correlated forecasts and verifying observation is the TRANSNORMAL model which generates
multivariate normal deviates which are then transformed into weather variables via their
cumulative distribution. <strong>The</strong> transormal model can relate with closed form expressions
quantitative, category and probability forecasts
CLIMATE PREDICTIONS differ from weather forecasts in several ways instead of being for a
single location at one time, they are often valid for a large region and an extended valid time
thus they are often expressed as distributions. Plus generally because of longer lead times,
they generally have lower skill and accuracy. Gringorten (1952) proposed an S score to verify
probability density forecasts. Plus synthetic verification indicates it is quite sensitive to low skill
forecasts.<strong>The</strong> S score was able to show that one year ahead seasonal temperature analog
forecasts forecasts produced by ETAC had significant improvement over chance.
CFARC MODEL a global extension of the transnormal model generates weather forecasts and
observation on a global scale thus providing a normative model for judging significance of such
things as widely separated telaconnections and global climate change.
CLIMATE CHANGE VERIFICATION: By using observations from one period as a prediction of
a later period, climate change can be verified. We note from experience that the climate is
always changing thus the question: has the climate changed is trivial since the answer is
always Yes! <strong>The</strong> real question is has the climate changed in an important manner?
A Glossary of over 28 verification terms is included. For example:
skill: takes into account the difficulty of the forecast in accessing the ability of a forecaster or
forecast method. Skill attempts to compare verification of easy forecasts with hard forecasts.
Generally, accuracy will be lower when: 1. the forecast event is rare, 2. data is sparse or
missing, or 3. the weather situation is complex; it does not fall into an easily recognized pattern.
See Skill Score.
2 ROC
Verification methodology using Relative Operating Characteristics (ROC), is derived from
signal detection theory. This methodology is intended to provide information on the
characteristics of systems upon which management decisions can be taken.(WMO2000)
Skill Score, SS: Is a generalized form to convert any score into a fractional improvement over
some standard. Thus: SS = [S(f) - S(s)] / [S(p) - S(s)] where S(f) is a any score based on
forecasts, S(s) is the score based on a standard such a persistence or climatology or chance ,
and S(p) is the perfect score. For forecasts better then the standard, SS has the range [0, 1]
with 0 when forecasts have the same accuracy as the standard and 1 when when forecasts
are∞the forecasts are perfect. SS is negative with range to - less than standard. When the
perfect score is zero as in the Brier score or MSE, it simplifies to: SS= 1 - S(f)/S(s). WARNING:
When the denominator can become small, skill scores are subject to considerable variability.
73