Predictability of weather regimes in ECMWF Forecasts - European ...

Predictability of weather regimes
in ECMWF Forecasts
Dan Rowlands, Warwick Norton
Cumulus, PCE Investors, London

Cumulus
• Hedge fund founded in 2006.
• $165m assets under management (pension funds, fund of funds, individual
investors).
• Two funds – Cumulus Energy, Cumulus Fahrenheit (weather derivaPves)
• Returns uncorrelated with other asset classes
• Team of 9 traders/analysts – recently joined aTer D.Phil at Oxford working on
climatepredicPon.net
• Max charge ECMWF user – makes use of determinisPc, EPS, monthly,
hindcasts, seasonal, reanalysis.
• We try to validate everything we use (run daily set of validaPon staPsPcs)

European energy markets and weather
• German power
– demand sensiPve to temperature (parPcularly cold temperatures)
– supply sensiPve to high temperature (can restrict use of Nuclear), wind
(29 GW of capacity), solar (24 GW of capacity), precip for hydro in the
Alps
• Nordic Power
– demand sensiPve to temperature
– hydro supplies 61% Norway, 33% Sweden electricity (need to manage
water stored in reservoirs, model snow accumulaPon/melt etc)
– Danish wind
• UK Power & Gas
– demand sensiPve to temperature, cloud cover
– Onshore & offshore wind

European energy markets and weather
• Consequently there is a huge price sensiPvity to the weather.
• Example from the EEX market.

Outline
• Weather regimes
• A recent example – Apr 2012
• Can we detect flow dependent skill/predictability?
– IniPal condiPons – regime/MJO
– Stratospheric forcing
• Thoughts on hindcast methodology
– Surface temperature calibraPon
• Conclusions

Weather Regimes
• Simplified way interprePng large
scale synopPc situaPon.
• Standard methodology using
clustering algorithm in reduced
dimension space using z500 from
ERA‐Interim over 20N‐80N and
90E‐60W.
• AddiPon of UKR (winter) and CLIMO
regimes.
• Set of regimes for each month to
account for seasonality.
CLIMO regime useful for transiPon days
NAO‐: NegaPve North AtlanPc OscillaPon
NAO+: PosiPve North AtlanPc OscillaPon
SB: Scandinavian Blocking
AR: AtlanPc Ridge
UKR: UK Ridge
CLIMO: Climatology

Weather Regimes (seasonality)
Winter
Spring

Weather Regimes (seasonality)
UKR/SB move west
through the
seasonal cycle
Spring
Summer

Frequency
0 10 20 30 40 50
Weather Regime DuraPon
era−interim z500 Jan NAO− 6.8 days
era−interim z500 Jan NAO+ 5.5 days
Frequency
0 10 20 30 40 50
0 10 20 30 40 50
0 10 20 30 40 50
Regime Duration
Regime Duration
era−interim z500 Jan SB 3.75 days
era−interim z500 Jan AR 4.63 days
Extended winter ‐
NDJFM
Frequency
0 10 20 30 40 50
Frequency
0 10 20 30 40 50
0 10 20 30 40 50
0 10 20 30 40 50
Regime Duration
Regime Duration
era−interim z500 Jan UKR 3.77 days
era−interim z500 Jan CLIMO 2.03 days
Frequency
0 10 20 30 40 50
Frequency
0 10 20 30 40 50
0 10 20 30 40 50
0 10 20 30 40 50
Regime Duration
Regime Duration

Weather Regimes – NAO+ DJF
t2m Mean t2m 10 th PercenPle t2m 90 th PercenPle

Weather Regimes – NAO‐ DJF
t2m Mean t2m 10 th PercenPle t2m 90 th PercenPle

Weather Regimes – SB DJF
t2m Mean t2m 10 th PercenPle t2m 90 th PercenPle

Weather Regimes – UKR DJF
t2m Mean t2m 10 th PercenPle t2m 90 th PercenPle

2011/2012 regimes
Temperature (deg C)
−4 −2 0 2 4 6 8 10 12 14
London tavg 20111201 − 20120430
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!
!
!
!
!
!
NAO−
NAO+
SB
AR
UKR
CLIMO
Dec Jan Feb Mar Apr
Date

LimitaPons of weather regimes – Jan 2011
Cumulative Prob
0.0 0.2 0.4 0.6 0.8 1.0
00Z eps z500 20110108
! ! ! ! ! ! ! ! ! ! !
11 10 9 8 7 6 5 4 3 2 1
Lead (days)
• All EPS forecasts from late December 2010 indicated correct NAO‐ regime.
• But, big shiT in London tavg around lead 6 (3 rd Jan).

Lead 9
Lead 6
LimitaPons of weather regimes – Jan 2011
20 30 40 50 60 70 80
20 30 40 50 60 70 80
350
300
100
−150
−50
250
−200
200
50
−100
150
−50
−100
−80 −60 −40 −20 0 20 40 60
200
ECMWF 00Z eps z500 2010123100 08 − 08 Jan lead = 9 day cor= 0.873
−250
−100
−150
−50
50
200
50
150
ECMWF 00Z eps z500 2011010300 08 − 08 Jan lead = 6 day cor= 0.926
300
350
250
0
−100
50
50
100
100
0
−50
−50
100
0
150
100
OperaPonal Analysis
OperaPonal Analysis
Lead 3
20 30 40 50 60 70 80
−80 −60 −40 −20 0 20 40 60
−300
−100
−150
−200
−50
250
200
0
150
ECMWF 00Z eps z500 2011010600 08 − 08 Jan lead = 3 day cor= 0.96
300
400
350
−150
−80 −60 −40 −20 0 20 40 60
−100
50
0
150
−50
50
100
100
150
50
ShiT from N/NE to SW flow
for London in forecasts aTer
lead 6.

April 2012
• Dominated by AR regime (19/30 days).
• Coldest April for 23‐years (UKMO).
• Forecasts all struggled keeping the cold air over the UK.
London tavg 20120401 − 20120430
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !
Temperature (deg C)
4 6 8 10 12 14
!
!
!
!
!
!
NAO−
NAO+
SB
AR
UKR
CLIMO
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Date

April 2012‐ Monthly 20120329
Monthly days 12‐18 (9‐15 th April)
OperaPonal Analysis

April 2012‐ Monthly 20120402
Monthly days 19‐25 (20 th ‐26 th April)
Forecast couldn’t persist the AR, preferring to develop a
SB like panern.
OperaPonal Analysis

ECMWF Monthly Forecasts
• Issued every Monday/Thursday (as of Nov 2011).
• 51 ensemble members out to 32 days.
• Every Thursday 18‐year 5 member hindcast produced.
• We have been trying to start to understand state/flow dependent skill/
predictability using the hindcast data set.
• Considered all forecasts from monthly system since resoluPon change
~Feb 2010 up to April 2012.
• ~2000 forecasts from the hindcast set and ~140 from the monthly. Focus
on hindcasts iniPally.

Hindcast skill through Pme
Days 5‐11
Days 12‐18
Days 19‐25
Days 26‐32
Pattern Correlation
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 Pattern Correlation 1992021100 − 2011032900 rolling 20 mean
1992 1994 1996 1998 2000 2002 2004 2006 2008 2010
Forecast Start Date
Panern correlaPon of the ensemble mean with ERA‐interim.
N AtlanPc/European Domain (20‐80N, 90E‐60W).
Days 11‐18 and 19‐25 have ~4 Pmes variance in performance.

Seasonal cycle of skill
Days 5‐11
Days 12‐18
Days 19‐25
Days 26‐32
Pattern correlation
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Month
Higher skill in winter, especially days 12‐18 and 19‐25,
somewhat driven by persistence in winter.

Monthly Forecast Skill – Feb 2010 onwards
Days 5‐11
Days 12‐18
Days 19‐25
Days 26‐32
Pattern correlation
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
monthly z500 Pattern correlation 2010021100 − 2012041900 rolling 10 mean
Feb Jun Oct Feb Jun Oct Feb
Forecast Start Date
Skill generally higher than hindcasts, likely due to bener
filter of the ensemble mean from larger ensemble size.

State/flow dependent skill
• Can we find parPcular regimes/states where skill is higher/lower in the
forecasts? Or is skill simply a random process?
• We have started with a few basic hypotheses, for which we have
disaggregated forecasts.
– IniPal and verifying regime linking through to regime transiPons.
– MJO strength and phase linking to tropical forcing.
– Stratospheric forcing in iniPal condiPons.
• Focus on winter forecasts (Nov‐March) where signals/teleconnecPons are
strongest and days 12‐18 and 19‐25.
• Ensemble size of hindcast set (5) prevents sensible applicaPon of probabilisPc
scores – consider the ensemble mean and most frequent regime verifying.

Skill by iniPal regime – days 12‐18
cor
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 cor winter days 12 − 18
918 176 225 145 209 163
−60 −40 −20 0 20 40 60
% difference on Overall
overall
ir.NAO−
ir.NAO+
ir.SB
ir.AR
ir.UKR
fcst.highest
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 fcst.highest winter days 12 − 18
918 176 225 145 209 163
−60 −40 −20 0 20 40 60
% difference on Overall
overall
ir.NAO−
ir.NAO+
ir.SB
ir.AR
ir.UKR
NAO‐ NAO+ SB AR UKR NAO‐ NAO+ SB AR UKR
Panern CorrelaPon of
ensemble mean
Blue = Forecast staPsPc
% improvement vs full set
Bars show 10‐90% sampling uncertainty
% of correctly
forecasted regime

Skill by iniPal regime – days 19‐25
cor
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 cor winter days 19 − 25
918 176 225 145 209 163
−60 −40 −20 0 20 40 60
% difference on Overall
overall
ir.NAO−
ir.NAO+
ir.SB
ir.AR
ir.UKR
fcst.highest
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 fcst.highest winter days 19 − 25
918 176 225 145 209 163
−60 −40 −20 0 20 40 60
% difference on Overall
overall
ir.NAO−
ir.NAO+
ir.SB
ir.AR
ir.UKR
NAO‐ NAO+ SB AR UKR NAO‐ NAO+ SB AR UKR
Panern CorrelaPon of
ensemble mean
Blue = Forecast staPsPc
% improvement vs full set
Bars show 10‐90% sampling uncertainty
% of correctly
forecasted regime

Skill by verifying regime – days 12‐18
cor
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 cor winter days 12 − 18
918 179 229 150 207 153
−60 −40 −20 0 20 40 60
% difference on Overall
overall
vr.NAO−
vr.NAO+
vr.SB
vr.AR
vr.UKR
fcst.highest
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 fcst.highest winter days 12 − 18
918 179 229 150 207 153
−60 −40 −20 0 20 40 60
% difference on Overall
overall
vr.NAO−
vr.NAO+
vr.SB
vr.AR
vr.UKR
NAO‐ NAO+ SB AR UKR NAO‐ NAO+ SB AR UKR
Panern CorrelaPon of
ensemble mean
Blue = Forecast staPsPc
% improvement vs full set
Bars show 10‐90% sampling uncertainty
% of correctly
forecasted regime

Skill by verifying regime – days 19‐25
cor
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 cor winter days 19 − 25
918 184 230 154 197 153
−60 −40 −20 0 20 40 60
% difference on Overall
overall
vr.NAO−
vr.NAO+
vr.SB
vr.AR
vr.UKR
fcst.highest
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 fcst.highest winter days 19 − 25
918 184 230 154 197 153
−60 −40 −20 0 20 40 60
% difference on Overall
overall
vr.NAO−
vr.NAO+
vr.SB
vr.AR
vr.UKR
NAO‐ NAO+ SB AR UKR NAO‐ NAO+ SB AR UKR
Panern CorrelaPon of
ensemble mean
Blue = Forecast staPsPc
% improvement vs full set
Bars show 10‐90% sampling uncertainty
% of correctly
forecasted regime

Persistent forecasts – days 12‐18
IniPal and verifying regime are the same
cor
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 cor winter days 12 − 18
918 61 73 25 79 32
−60 −40 −20 0 20 40 60
overall
iRp.NAO−
iRp.NAO+
iRp.SB
iRp.AR
iRp.UKR
% difference on Overall
fcst.highest
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 fcst.highest winter days 12 − 18
918 61 73 25 79 32
−60 −40 −20 0 20 40 60
overall
iRp.NAO−
iRp.NAO+
iRp.SB
iRp.AR
iRp.UKR
% difference on Overall
NAO‐ NAO+ SB AR UKR NAO‐ NAO+ SB AR UKR
Panern CorrelaPon of
ensemble mean
Blue = Forecast staPsPc
% improvement vs full set
Bars show 10‐90% sampling uncertainty
% of correctly
forecasted regime

Regime transiPons – days 12‐18
IniPal and and verifying regime are are different
cor
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 cor winter days 12 − 18
918 115 152 120 130 131
−60 −40 −20 0 20 40 60
overall
iRd.NAO−
iRd.NAO+
iRd.SB
iRd.AR
iRd.UKR
% difference on Overall
fcst.highest
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 fcst.highest winter days 12 − 18
918 115 152 120 130 131
−60 −40 −20 0 20 40 60
overall
iRd.NAO−
iRd.NAO+
iRd.SB
iRd.AR
iRd.UKR
% difference on Overall
NAO‐ NAO+ SB AR UKR NAO‐ NAO+ SB AR UKR
Panern CorrelaPon of
ensemble mean
Blue = Forecast staPsPc
% improvement vs full set
Bars show 10‐90% sampling uncertainty
% of correctly
forecasted regime

MJO
+ days 5‐11
Phase 3
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim z500 3 MJO phase 3 day 5 − 11 n = 444
NAO− NAO+ SB AR UKR CLIMO
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim z500 3 MJO phase 3 day 12 − 18 n = 444
NAO− NAO+ SB AR UKR CLIMO
+ days 12‐18
Regime
Regime
+ days 19‐25
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim z500 3 MJO phase 3 day 19 − 25 n = 444
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim z500 3 MJO phase 3 day 26 − 32 n = 444
+ days 26‐32
NAO− NAO+ SB AR UKR CLIMO
NAO− NAO+ SB AR UKR CLIMO
Regime
Cassou 2008 found phase 3 + 6 iniPate NAO+ and NAO‐ respecPvely in winter
(Nov‐March here for ERA‐Interim).
Regime

MJO
+ days 5‐11
Phase 6
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim z500 3 MJO phase 6 day 5 − 11 n = 377
NAO− NAO+ SB AR UKR CLIMO
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim z500 3 MJO phase 6 day 12 − 18 n = 377
NAO− NAO+ SB AR UKR CLIMO
+ days 12‐18
Regime
Regime
+ days 19‐25
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim z500 3 MJO phase 6 day 19 − 25 n = 377
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim z500 3 MJO phase 6 day 26 − 32 n = 377
+ days 26‐32
NAO− NAO+ SB AR UKR CLIMO
NAO− NAO+ SB AR UKR CLIMO
Regime
Cassou 2008 found phase 3 + 6 iniPate NAO+ and NAO‐ respecPvely in winter
(Nov‐March here for ERA‐Interim).
Regime

IniPal MJO phase – days 12‐18
cor
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 cor winter days 12 − 18
918 344 53 73 100 75 74 71 70 58
−60 −40 −20 0 20 40 60
% difference on Overall
overall
iMJO.0
iMJO.1
iMJO.2
iMJO.3
iMJO.4
iMJO.5
iMJO.6
iMJO.7
iMJO.8
fcst.highest
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 fcst.highest winter days 12 − 18
918 344 53 73 100 75 74 71 70 58
−60 −40 −20 0 20 40 60
% difference on Overall
overall
iMJO.0
iMJO.1
iMJO.2
iMJO.3
iMJO.4
iMJO.5
iMJO.6
iMJO.7
iMJO.8
Panern CorrelaPon of
ensemble mean
Blue = Forecast staPsPc
% improvement vs full set
Bars show 10‐90% sampling uncertainty
% of correctly
forecasted regime

MJO – phase 7 + 8
+ days 12‐18 + days 19‐25
Phase 7
Phase 8
In days 12‐18 and 19‐25 aTer phase 7+8 tendency for NAO‐ (owing to iniPaPon in
phase 6 and Pmescale of MJO propagaPon?).

Stratospheric forcing
ERA‐interim DJF.
Coldest 10% of stratospheric
temperatures.
NAO+/AR response.
ERA‐interim DJF.
Warmest 10% of stratospheric
temperatures.
NAO‐ response.
Define through 100hPa temperatures poleward of 70N (Mark Baldwin, Exeter)

Stratospheric forcing ‐ regimes
ERA‐interim DJF
Coldest 10% of days,
NAO+/AR response
ERA‐interim DJF
Warmest 10% of days,
NAO‐ response.

Stratospheric forcing ‐ persistence
+ days 5‐11
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim t100 Jan −6.34 deg 10 % day 5 − 11 n = 295
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim t100 Jan −6.34 deg 10 % day 12 − 18 n = 295
+ days 12‐18
NAO− NAO+ SB AR UKR CLIMO
NAO− NAO+ SB AR UKR CLIMO
Regime
Regime
+ days 19‐25
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim t100 Jan −6.34 deg 10 % day 19 − 25 n = 295
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim t100 Jan −6.34 deg 10 % day 26 − 32 n = 295
+ days 26‐32
NAO− NAO+ SB AR UKR CLIMO
NAO− NAO+ SB AR UKR CLIMO
Regime
Regime
ERA‐Interim. Coldest 10% of stratospheric temperatures.
Avg duraPon ~ 9 days

Stratospheric forcing ‐ persistence
+ days 5‐11
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim t100 Jan 8.36 deg 90 % day 5 − 11 n = 295
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim t100 Jan 8.36 deg 90 % day 12 − 18 n = 295
+ days 12‐18
NAO− NAO+ SB AR UKR CLIMO
NAO− NAO+ SB AR UKR CLIMO
Regime
Regime
+ days 19‐25
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim t100 Jan 8.36 deg 90 % day 19 − 25 n = 295
Frequency of occurence
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Obs
Climo
era−interim t100 Jan 8.36 deg 90 % day 26 − 32 n = 295
+ days 26‐32
NAO− NAO+ SB AR UKR CLIMO
NAO− NAO+ SB AR UKR CLIMO
Regime
Regime
ERA‐Interim Warmest 10% of stratospheric temperatures.
Average duraPon ~ 11 days.
Persistence of NAO‐ regime.

Stratospheric forcing – warming events
ERA‐Interim
Hindcast
12‐18
19‐25
Model able to persist the NAO‐ panern (i.e. stratospheric anomaly?) during
warm stratosphere events (in a composite sense).
But, only seeing ~20 events given persistence – look at t100 skill?

Stratospheric forcing – cold events
ERA‐Interim
Hindcast
12‐18
19‐25
More complex regime picture for cold stratosphere events.
AR/UKR response, weakly picked up by model.
Sets up preference for NAO+ ‐> AR/UKR transiPon?

Stratospheric forcing skill scores – days 12‐18
cor
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 cor winter days 12 − 18
918 102 112
−60 −40 −20 0 20 40 60
% difference on Overall
overall
t100.high
t100.low
fcst.highest
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 fcst.highest winter days 12 − 18
918 102 112
−60 −40 −20 0 20 40 60
% difference on Overall
overall
t100.high
t100.low
Warm t100 Cold t100
Warm t100 Cold t100
Panern CorrelaPon of
ensemble mean
Blue = Forecast staPsPc
% improvement vs full set
Bars show 10‐90% sampling uncertainty
% of correctly
forecasted regime

Stratospheric forcing skill scores – days 19‐25
cor
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 cor winter days 19 − 25
918 102 112
−60 −40 −20 0 20 40 60
% difference on Overall
fcst.highest
0.0 0.2 0.4 0.6 0.8 1.0
hindcast z500 fcst.highest winter days 19 − 25
918 102 112
overall
t100.high
t100.low
overall
t100.high
t100.low
−60 −40 −20 0 20 40 60
% difference on Overall
Warm t100 Cold t100
Warm t100 Cold t100
Panern CorrelaPon of
ensemble mean
Blue = Forecast staPsPc
% improvement vs full set
Bars show 10‐90% sampling uncertainty
% of correctly
forecasted regime

Monthly Forecast Skill scores
Pattern Correlation
0.0 0.2 0.4 0.6 0.8 1.0
monthly z500 Pattern Correlation winter n = 71
Brier Skill Score
0.0 0.2 0.4 0.6 0.8 1.0
monthly z500 Brier Skill Score winter n = 71
5 − 11 12 − 18 19 − 25 26 − 32
5 − 11 12 − 18 19 − 25 26 − 32
Day range
Day range
Panern CorrelaPon of
ensemble mean
Winter Forecasts (71 in total), validaPng over 20N‐80N and 90E‐60W.
Bars show 10‐90% sampling uncertainty.
Brier Skill Score on
regime frequencies

Monthly Forecast Skill scores
Brier Skill Score on
individual regime
frequencies
Brier Skill Score
!0.2 0.0 0.2 0.4 0.6 0.8 1.0
monthly z500 Brier Skill Score winter n = 71
5 ! 11 12 ! 18 19 ! 25 26 ! 32
NAO!
NAO+
SB
AR
UKR
But, period not
necessarily
representaPve of
climatology (more
NAO‐, UKR, less
NAO+). So scores
may overstate skill.
Poor skill for SB
regime
Day range
Winter Forecasts (71 in total), validaPng over 20N‐80N and 90E‐60W.
Bars show 10‐90% sampling uncertainty.

Monthly Forecast – is it persistence?
Forecast ‐ solid
Persistence of day 1
regime ‐ dashed
Brier Skill Score
!0.4 !0.2 0.0 0.2 0.4 0.6 0.8 1.0
!
monthly z500 Brier Skill Score winter n = 71
!
!
!
NAO!
NAO+
5 ! 11 12 ! 18 19 ! 25 26 ! 32
!
Day range
Winter Forecasts (71 in total), validaPng over 20N‐80N and 90E‐60W.
Bars show 10‐90% sampling uncertainty.

Thoughts on hindcasts for calibraPon
• A key task for Cumulus is producing staPon forecasts for major ciPes
necessary for demand forecasts and weather derivaPves
• This requires calibraPon of model variables.
• Challenge how do to this with clear seasonality in some locaPons and
usually

Las Vegas Hindcast v EPS forecasts
lasvegas tmax box 1 day 1 Hindcast − EPS mean
Hindcast − EPS
EPS_5member − EPS
−3 −2 −1 0 1 2
−3 −2 −1 0 1 2
!
!
!
!
!
!
!
!!
!
!
!
!
!! ! !
!
! ! !
!
!
!
!
!
! ! !
!
!
!
!
!
!
Feb Apr Jun Aug Oct Dec Feb Apr Jun
!
Fcst
Climo
! !
!
!
!
!
Date
lasvegas tmax box 1 day 1 EPS_5member − EPS mean % out = 84.5 % target = 10 %
!
!
! ! !
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
! !
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
! !
• Large differences in tmax between
EPS and hindcast on the same date
(& tmin).
• Not explained by differences in
model climatology (blue).
• Not an ensemble size issue (>80% of
biases outside the 5‐95% range due
to sampling uncertainty from EPS
with 5 member ensemble).
• Should we expect them to be the
same? Different model cycle,
iniPalizaPon and ensemble
perturbaPon method.
• Similar finding for other locaPons.
!
!
!
0 10 20 30 40 50 60 70
ID by sorted sampling range

Conclusions
• We find weather regimes are a useful tool for to simplify the high dimensional output
produced by the EPS/monthly/seasonal system.
• Analysis of skill from hindcast set reveals that persistence of regimes (especially NAO+/‐) is
generally bener forecast than transiPons. We find linle influence from the MJO on skill, whilst
the presence (persistence) of stratospheric forcing is quite well captured by the model.
• BUT: monthly system limited by number of forecasts, hindcast system limited by ensemble
size hence large uncertainty and rather tentaPve conclusions (against the null hypothesis of
skill being a random process).
– Larger & longer hindcast ensembles? Welcome the extension to 20‐years in cycle 38r1.
– 5 ensemble members makes probabilisPc evaluaPon tricky.
– Look at insights from EPS for day 10 onwards given larger sample size.
• However, for site specific forecasts, given high sensiPviPes, regimes may be an
oversimplificaPon.
• We would welcome more research/clarity on impact of the different iniPalizaPon on hindcast
surface temperature forecasts.

April 2012
• Dominated by AR regime (19/30 days).
• Coldest April for 23‐years (UKMO).
• Forecasts all struggled keeping the cold air over the UK.
Analysis
Forecast

December 2011
RMM2
−4 −3 −2 −1 0 1 2 3 4
!
!
!
!
!
monthly 2011120100 MJO RMM1&2 cor = 0.826 rmse/clim_rmse = 0.57
8
1
day1
day5
day10
day15
day20
7
!
!
!
!
!
!
!
! !
! !
!
!
!
!
! ! ! !
! !!
!
!
!
!
!
! ! ! ! !
!
! !
!
!
!!
!!
! !! ! !
!
!!
!
! !
! !!!!!
! !!
! ! ! !
! ! ! !
!
!
!
!!
! !
!
!
!
!
! !
!
!
!
!
! !!
!
!
!
!
!
!
!
! !
!
!
!
!
!
!
2 3
6
5
4
−4 −3 −2 −1 0 1 2 3 4
RMM1
Not a parPcularly “interesPng” MJO event, but did persistent convecPon in the
Indian Ocean give rise to persistent NAO+/UKR regime?

December 2011 – Monthly 20111512

December 2011
Week 3 20111201 Week 3 20111201 Week 2 20111215
Forecasts performed well in general, capturing the persistence of the NAO+
regime.