(Jean Christopher Lambert, BIRA) - 2.6Mb pdf - QA4EO

QA4EO Test Case II – Atmospheric Composition and Climate
QA4EO and the Remote Sensing
of Atmospheric Composition
Jean-Christopher LAMBERT
Belgian Institute for Space Aeronomy (IASB-BIRA), Brussels, Belgium
1 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

QA4EO and the Remote Sensing
of Atmospheric Composition
1. Scientific/decision/policy thematic domains
2. Sources and characteristics of data
3. Quality requirements
4. Impact of not achieving requirements
5. Feedback and perspectives
2 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

1. Scientific/decision/policy
thematic domains and goals
3 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

Atmospheric Remote Sensing and SBAs
space weather
CFCs, HCFCs, HFCs
UV radiation
HCHO
CHOCHO
Biodiv.
Disasters
5
4
3
2
Health
ClONO 2
NO
SO 2
NO 2
BrO
HNO 3
Agric.
1
Energy
CO
OClO
surface
Ecosys.
Climate
O 3
aerosols
CO 2
Weather
Water
H 2 O
NH 3
HCl
HF
COF 2
temperature
N 2 O
CH 4
4 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

2. Data
Sources and characteristics
5 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

Integrated Global Observing Strategy
Satellites
Aircrafts and balloons
SAOZ-bal
132 flights
Ground networks
6 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

NETWORK FOR THE DETECTION OF
ATMOSPHERIC COMPOSITION CHANGE (NDACC)
Cooperating Networks
http://ndacc.org
HATS
7 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

SOUNDER
MISSION
SATELLITE MISSIONS FOR ATMOSPHERIC COMPOSITION (1978-2020)
TOMS Nimbus 7
METEOR 3
ADEOS 1
Earth Probe
SBUV Nimbus 7
SBUV/2,
NOAA-9
SSU/AMSU,
TOVS
NOAA-11
NOAA-14
NOAA-16
NOAA-17
NOAA-N/18
2020
2019
2018
2017
2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
2000
1999
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
1988
1987
1986
1985
1984
1983
1982
1981
1980
1979
1978
AMSU-A, HIRS/4 NOAA-N1/19
SSBUV STS I I I I I I I I
SOLSE/LORE STS 87 & 107 I I
OMPS, CrIS, ATMS NPP
OMPS, CrIS, ATMS NPOESS
SBUS, TOU, IRAS, Feng-Yun-3A
MWTS, MWHS, Feng-Yun-3B
MERSI
Feng-Yun-3x
SAM II Nimbus 7
SAGE I
AEM-B
SAGE II
ERBS
SAGE III
METOR 3M
LITE STS-64 I
CALIPSO
CALIPSO
LIMS
SAMS
UV / VIS / IR
Nimbus 7
SME
GRILLE STS 9/45 I I
ATMOS 1/2/3/4 STS 17/45/56/66 I I I I
CLAES
HALOE
ISAMS
UARS
MLS
MAHRSI STS 66 & 85 I I
MOPITT
SABER
MLS
HIRDLS
OMI
TES
SOFIE
EOS Terra
TIMED
AIM
POAM II SPOT 3
POAM III SPOT 4
BUV
EXOS-C
LAS
(Ohzora)
ILAS
ADEOS
IMG
(Midori)
ILAS-II
ADEOS-II
TANSO
GOSAT
SMILES
Kibo JEM/ISS
ORA
EURECA
CRISTA 1/2 STS 66 & 85 I I
GOME
ERS-2
SCIAMACHY
GOMOS
MIPAS
GOME-2, IASI,
ENVISAT-1
MetOp-A
AMSU-A, HIRS/4
GRAS, MHS
TropOMI
OZONE/ISTOK
SMR
OSIRIS
ACE FTS
ACE MAESTRO
STEAM
SWIFT
EOS Aura
MetOp-B
MetOp-C
Sentinel-5P
MIR/Priroda
Odin
SCISAT-1
STEAM
Chinook
Status May 2011 Sounding strategy: nadir Sun/Moon occultation Spectral range: UV UV/VIS/NIR
Compiled by CEOS ACSG and the NDACC Satellite WG limb 8 stellar occultation QA4EO Workshop, RAL, Harwell IROxford, UK, Oct. VIS/IR 18-20, 2011
Details via http://www.ndacc.org Satellite WG nadir/limb multi-target MW multi-sensor

Information content of remote sensing data
Air masses probed by GOME, Envisat and NDACC ground-based instruments
9 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

Information content of remote sensing data
Nadir vertical sensitivity (GOME-1/2, SCIAMACHY, OMI…)
NO 2 COLUMN RETRIEVAL
O 3 COLUMN RETRIEVAL
O 3
NO 2
10 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

3. Quality requirements
What could be a valid Quality Indicator
(providing evidence of fitness for purpose)
11 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

Quality requirements
Cardinal requirements
Error bars on species & on height
Vertical and horizontal resolution
Observation frequency
Info primarily from observation
Documentation, metadata
Specific requirements
Air quality
Climate: GHG, ozone, water vapour, clouds, aerosols…
Polar ozone, UV radiation, circulation…
Particulate matter, allergens, health hazards
Volcanic hazards to aviation
Corrosion and abrasion, degradation by UV…
Radiative transfer, atmospheric correction…
…
12 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

[GOME - SAOZ]/SAOZ Total Ozone (%)
Evolution of GOME total ozone data accuracy since 1995…
40
SZA dependence is a QI.
30
20
10
0
-10
-20
-30
(GOME-SAOZ)/SAOZ [%]
Tropic of Capricorn
Northern Mid-latitudes
60° North
Arctic Polar Circle
Northern Polar Latitudes
1995
GDP 1.20
20 30 40 50 60 70 80 90 100
20
15
10
5
0
-5
-10
Solar zenith angle of GOME (deg)
GDP 2.0
1996
GDP 2.7 / 3.0 / 4.0
1999 / 2003 / 2006
GDP 4.1 / 5.0
2008 / 2011
-15
-20
All latitudes excepted Antarctica
Vertical error bars: standard deviation (2σ)
20 30 40 50 60 70 80 90 100
Solar zenith angle of GOME [deg.]
13 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

GOME NADIR O3P VERTICAL SMOOTHING
Vertical resolution and altitude registration as QI
Meijer et al., JGR 2006
Meijer et al., JGR 2006
14 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

GOME Nadir O 3 P vertical smoothing error as QI
30-40 km
20-25 km
2-10 km
Lambert et al., 2004; De Clercq et al., 2006
15 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

GOME NADIR O 3 P INFORMATION CONTENT
Averaging kernels and derived quantities as QIs
Preliminary version R1
De Clercq et al., ESA SP-636, 2007
16 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

GOME NADIR O 3 P INFORMATION CONTENT
Averaging kernels and derived quantities as QIs
Improved version R2
De Clercq et al., ESA SP-636, 2007
17 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

GOME NADIR O 3 P INFORMATION CONTENT
vs. Altitude and Time: Northern Middle Latitudes
De Clercq et al., ESA SP-636, 2007
18 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

4. Impact of not achieving
requirements
19 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

Impact of not achieving requirements
No use of the data
Fictitious patterns and temporal variations
Inaccurate assessment of (polar) ozone loss/recovery
Bias towards specific conditions (e.g., clouds, land…)
Too noisy => many more years needed to detect trends
Wrong trends, or inaccurate trends
Erroneous interpretation of trends and events
Erroneous conclusions on data quality
Erroneous conclusions on climate-atmosphere interactions
Etc.
‣ Clear impact on scientific results, on assessments, on
protocol monitoring, on decision making…
20 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

[GOME - SAOZ]/SAOZ Total Ozone (%)
Evolution of GOME total ozone data accuracy since 1995…
40
30
20
10
0
-10
-20
-30
Tropic of Capricorn
Northern Mid-latitudes
60° North
Arctic Polar Circle
Northern Polar Latitudes
1995
GDP 1.20
20 30 40 50 60 70 80 90 100
Solar zenith angle of GOME (deg)
GDP 2.7 / 3.0 / 4.0
1999 / 2003 / 2006
(GOME-SAOZ)/SAOZ [%]
20
15
10
5
0
-5
-10
GDP 2.0
1996
GDP 4.1 / 5.0
2008 / 2010
-15
-20
All latitudes excepted Antarctica
Vertical error bars: standard deviation (2σ)
20 30 40 50 60 70 80 90 100
Solar zenith angle of GOME [deg.]
21 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

How valid are uncertainty estimates given in a
data file / derived from a data comparison
Method described in Section 4.1 of Cortesi et al., ACP 2007
22 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

How compact are tracer-tracer correlations and hydrogen budgets
Lambert et al., ISSI 2011 (with figures adapted from von Clarmann et al., AMT 2009)
23 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

5. Feedback and perspectives
on QA4EO role and implementation
24 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

Feedback on QA4EO Role
As usually, QA is considered as crucial, but remains
given poor attention and resource.
QA4EO advertised in CEOS WGCV ACSG, in NDACC, in
GMES Atmospheric Service, in SPARC, in UNEP/WMO
ORM… and within space agencies and validation groups.
QA4EO principles are agreed to be key to success. But
QA4EO framework judged from “too simple” to “too
demanding”.
Level of detail needed is not clear. Costs can grow
exponentially => Guidance is requested.
What constitutes a valid Quality Indicator is unclear,
especially if it has to be “traceable to possibly SI”.
Concerns with the ambiguous concept of compliance.
Sometimes confusion with ISO and with European
Directive INSPIRE
25 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

Feedback on QA Strategy Implementation
Principles implemented – at least partly and/or formally –
in ESA projects (GOME, GSE, GECA, CCI…), NASA
ROSES, EUMETSAT O3M-SAF, NDACC WGs, GMES
Atmospheric Service etc.
Degree and quality of implementation very unequal from
one team to another. In many teams and communities,
systematic characterisation and error budget studies
have been ongoing effort for years. But for a few
others, EO is a “beautiful picture” science and does not
need QI as long as detection of events is a validation!
Very broad range of user requirements, sometimes
contradicting, not always clear or available to the
retrieval teams. URDs need translation into Cal/Val Req.
Cyclic renewal of AC&C RS community => Sustainability
of QA4EO type implementation beyond the 4-year
horizon
26 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

Ozone_cci Product
Validation Plan
27 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

Feedback on QA Strategy Implementation
A QI can’t be valid/tested on every satellite pixel and
assumptions are to be made. So, the same product can be
QA4EO compliant globally but not at pixel scale. Idem
for multi-mission data sets.
Although standards exist (VIM, GUM…), terminology is
not consistently used (e.g. “accuracy”).
Terminology taken from metrology not always applicable,
specially to models.
QA practices with different quality live in parallel.
Bias and noise introduced by neglecting smoothing and
sampling errors can spoil the value of validation efforts
and lead to misinterpretations of the data.
Information content studies based on AKs are
encouraged. But AK availability is an issue.
28 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011

End-to-end QI
End-to-end validation is recommended in theory, e.g. in
formal validation protocols. In the facts, most of
components of a production chain are characterised for
level-1 up to level-2, although unequally and without
coordination. Documentation is scattered and incomplete.
Propagation of uncertainties from Level-0 to Level-2 is
established practice. Propagation of uncertainties up to
Level-3/4 not straightforward, not always feasible.
Information content is an known issue in the DA
community but the implementation of improved
observation operators is not a priority.
Information content aspects of merged data sets
29 QA4EO Workshop, RAL, Harwell Oxford, UK, Oct. 18-20, 2011