Reconstructions

Quantifying uncertainties
associated with calibrating proxy
data against instrumental
records – an ensemble approach
Ailie Gallant 1 , Jöelle Gergis 1 and Karl Braganza 2
1 School of Earth Sciences
University of Melbourne, Australia
2 National Climate Centre
Australian Bureau of Meteorology
11 IMSC, 12 – 16 July 2010

Outline
1. Common methodological practices for
developing reconstructions
2. Calibrating to a single period – musings and
questions arising
3. An example using a multi-proxy
reconstruction of southeast Australian
rainfall and streamflow

Palaeoclimate reconstructions – common practice
• Common practice is to calibrate against
instrumental records
– Use a portion of the instrumental record for calibration e.g.
apply regression
– Retain the remaining portion for independent verification of the
record
– The suitability and accuracy of the reconstruction is generally
assessed using a suite of statistics (e.g. correlations, sign test,
RMSE, RE, CE etc.)
– These statistics are used to give an indication of the
confidence we place in the accuracy of the reconstruction
While the accuracy of the reconstruction is stated in terms of
these validation statistics, there is no statement about the
confidence in the numbers themselves…

Musings and questions
If the choice of calibration/verification period and
the lengths of these periods is arbitrary…
1. How dependent is the final reconstruction to this arbitrary
calibration period? Magnitude of sampling error?
2. How is the validation and subsequent “accuracy” of the
reconstruction affected by the arbitrary choice of
calibration/verification period?
3. The final reconstruction is often calibrated against the
entire instrumental record, are we accurately estimating
the regression coefficients? Are we tuning the
reconstruction?

Should we use single/split period calibration?
Proxies may be accepted/rejected as suitable for climate
reconstruction based on these arbitrary choices, with no
estimate as to how representative these statistics actually are
and no estimation of the possible size of the sampling error
associated with calibrating to one period.

An example: Rainfall reconstruction for SE Australia

Problems with the use of single/split calibration
• Period of overlap with observations from 1900 – 1988
• Split the time series in two:
– Early period calibration (1900 – 1944)
– Late period calibration (1945 – 1988)
The verification statistics and the perceived accuracy of the reconstruction
was strongly dependent on the choice of calibration/verification period

An ensemble approach to estimating uncertainty
• Employed bootstrapping to generate an ensemble of
reconstructions and verification statistics
• Allows some quantification of sampling error in
i. the verification statistics
ii. the reconstruction associated with calibration (i.e. the
dependence of the reconstruction on calibration period)
• Random calibration intervals were generated using
approximately half the data – selected in decadal blocks to
maintain variability on the decadal and shorter scales
• 10,000-member ensemble generated, decades used for
generation of ensemble had a uniform distribution

Errors associated with calibration

Reconstructions
Annual reconstruction
Decadal reconstruction
• Can take the median of the
ensemble as the “best
estimate” of the reconstruction
– resulting correlation very
similar to “full period”
calibration (both 0.57), but
with better estimates of other
measures of reconstruction
accuracy
• Perhaps a sign that using full
period calibration is not tuning
to 20 th century for this case?
• Better to treat the
reconstruction ensemble as a
whole – also allows
probabilistic estimates of the
likelihood of a past climate

River Murray streamflow reconstruction:
Potential amplification of errors associated
with calibration

Probabilistic estimate of past climate
• By using an ensemble, we can estimate the
likelihood of a past event occurring rather than
relying on en estimate from one realisation of a
reconstruction
• Example, streamflow reconstruction: The present
day decadal-scale drought exceeds the median
“best estimate”, but is not outside the range of the
ensemble
• 13% chance of droughts from 1783 – 1988
exceeding present day drought (according to the
reconstruction)

Summary
• Serious weight is given to how we perceive the accuracy
of a reconstruction by verifying to a particular period
• The arbitrary choice of calibration/validation period can
significant affect this perception and can also lead to the
introduction of sampling errors in the reconstruction
• Ensemble approach means we can have a “best estimate”
while still retaining information on sampling errors
• Can obtain a distribution of possible reconstructions back
through time – allows from a probabilistic estimate of the
likelihood of that reconstruction

THANK YOU
agallant@unimelb.edu.au