EnviroInfo 2004: DEM-AQ: Data Exchange Module for Air Quality data
EnviroInfo 2004: DEM-AQ: Data Exchange Module for Air Quality data
EnviroInfo 2004: DEM-AQ: Data Exchange Module for Air Quality data
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>EnviroInfo</strong> <strong>2004</strong> (Geneva)<br />
Sh@ring – <strong>EnviroInfo</strong> <strong>2004</strong><br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3<br />
<strong>DEM</strong>-<strong>AQ</strong>: <strong>Data</strong> <strong>Exchange</strong> <strong>Module</strong> <strong>for</strong> <strong>Air</strong> <strong>Quality</strong><br />
<strong>data</strong><br />
Wim Mol 1<br />
Abstract<br />
Since 1998 the European Topic Centre <strong>for</strong> <strong>Air</strong> and Climate Change (ETC/ACC 2 )<br />
and its predecessor the European Topic Centre on <strong>Air</strong> <strong>Quality</strong> has placed the <strong>DEM</strong>-<br />
<strong>AQ</strong> (<strong>Data</strong> <strong>Exchange</strong> <strong>Module</strong> <strong>for</strong> <strong>Air</strong> <strong>Quality</strong> <strong>Data</strong>) at the disposal of air quality <strong>data</strong><br />
suppliers. The <strong>DEM</strong>-<strong>AQ</strong> is a tool <strong>for</strong> <strong>data</strong> exchange between countries supplying air<br />
quality <strong>data</strong> and the European central <strong>data</strong>base <strong>for</strong> air quality <strong>data</strong>: AIRBASE.<br />
The <strong>DEM</strong>-<strong>AQ</strong> supports collection of air quality <strong>data</strong> and associated meta <strong>data</strong> required<br />
by the various legal and moral obligations. These obligations are: Council<br />
Decision 97/101/EC (‘<strong>Exchange</strong> of In<strong>for</strong>mation’, (EoI)), EEA-Euro<strong>Air</strong>net programme<br />
and EMEP Task Force on Measurement and Modelling (TFMM).<br />
1. History<br />
Based on a review of the old <strong>Exchange</strong> of In<strong>for</strong>mation (EoI) <strong>data</strong>bases APIS and<br />
GIRAFE, experience of <strong>data</strong> suppliers, the EoI Decision (97/101/EC), and the EEA<br />
<strong>data</strong> needs, it was recommended by the EEA in 1995 to develop a new air quality<br />
in<strong>for</strong>mation system. Under the yearly EEA subventions, the then ETC-<strong>AQ</strong> started in<br />
close collaboration with the Directorate General Environment of the European<br />
Commission (DGEnv) and EEA with the development of the new system called<br />
AIRBASE. The objective of the AIRBASE in<strong>for</strong>mation system was <strong>for</strong>mulated as:<br />
1 European Topic Center <strong>for</strong> <strong>Air</strong> and Climate Change (ETC/ACC)<br />
RIVM, National Institute <strong>for</strong> Public Health and the Environment<br />
P.O. Box 1, 3720 BA Bilthoven, the Netherlands<br />
Tel. +31 30 274 2378, Fax. +31 30 274 4433, email: wim.mol@rivm.nl<br />
http://etc-acc.eionet.eu.int/<br />
2 The European Topic Center <strong>for</strong> <strong>Air</strong> and Climate Change (ETC/ACC) ia a consortium of<br />
European institutes (RIVM, UBA-B, UBA-V, IIASA, NILU, AEAT, AUTh, CHMI, DNMI,<br />
NTUA, ÖKO, UEA, IEP, TNO) under contract of the European Environmental Agency<br />
(EEA).<br />
256
Offering the facilities needed to collect, validate, evaluate, store and visualise raw<br />
air quality <strong>data</strong> and statistics and (meta) in<strong>for</strong>mation on air quality monitoring networks<br />
and stations, and making these <strong>data</strong> available on different aggregated levels,<br />
<strong>for</strong> different in<strong>for</strong>mation users and through different distribution channels.<br />
AIRBASE is at the service of different target groups: experts, policy makers and the<br />
general public.<br />
The <strong>DEM</strong>-<strong>AQ</strong> was implemented to load AIRBASE with air quality in<strong>for</strong>mation.<br />
The first version of the <strong>DEM</strong>-<strong>AQ</strong> was released in 1998. The software is used to exchange<br />
air quality <strong>data</strong> (raw <strong>data</strong>, statistics and ozone exceedances) and meta in<strong>for</strong>mation<br />
on operational air quality networks, stations and measurement configurations<br />
in the framework of the EoI Council Decision 97/101/EC, the EEA-Euro<strong>Air</strong>net<br />
programme and the EMEP Task Force on Measurement and Modelling (TFMM).<br />
With the <strong>DEM</strong>-<strong>AQ</strong> the <strong>data</strong> supplier can add, modify and delete meta in<strong>for</strong>mation<br />
on air quality networks, stations and measurement configurations. The <strong>data</strong> supplier<br />
can also import <strong>data</strong> files with extensive error checking. The <strong>DEM</strong>-<strong>AQ</strong> supports also<br />
visualisation of the imported <strong>data</strong> and the transmission of the <strong>data</strong> <strong>for</strong> incorporation<br />
in AIRBASE. Yearly the <strong>DEM</strong>-<strong>AQ</strong> is pre-loaded with all the meta in<strong>for</strong>mation contained<br />
in AIRBASE and is send to the <strong>data</strong> suppliers and the <strong>data</strong> suppliers load the<br />
<strong>DEM</strong>-<strong>AQ</strong> again with the <strong>data</strong> of the last year. The <strong>DEM</strong>-<strong>AQ</strong> is used by almost 30<br />
counties: the EU Member States, EU-Accession countries, EFTA states and some<br />
other European countries: Albania, Bosnia and Herzegovina and FYROM (Former<br />
Yugoslavian Republic of Macedonia).<br />
AIRBASE now contains air quality <strong>data</strong> of more than 30 European countries, almost<br />
40 air quality parameters, 370 networks, 4000 stations and and more than<br />
15000 time series. AIRBASE has be made available to the public on Internet via the<br />
ETC/ACC Web site 3 (Mol 2003).<br />
2. Structure AIRBASE in<strong>for</strong>mation System<br />
AIRBASE is an Ingres RDBMS running on UNIX. Later on in <strong>2004</strong> AIRBASE will be<br />
converted in Oracle. Because the <strong>DEM</strong>-<strong>AQ</strong> (Access <strong>data</strong>base) has the same <strong>data</strong>model<br />
as AIRBASE, the <strong>DEM</strong>-<strong>AQ</strong> can be populated with all available meta<br />
in<strong>for</strong>mation on <strong>AQ</strong> networks, stations and measurement configurations including<br />
info on available time series from AIRBASE. The preloaded <strong>DEM</strong>-<strong>AQ</strong> is transmitted<br />
to the <strong>data</strong> suppliers in the countries. The complete <strong>data</strong> flow can be followed in<br />
Figure 1. The <strong>data</strong> supplier checks the meta in<strong>for</strong>mation by adding, modifying or<br />
deleting the meta <strong>data</strong> and imports the raw measurement <strong>data</strong> through several <strong>for</strong>mats<br />
of import files. The imported files are checked automatically by the <strong>DEM</strong>-<strong>AQ</strong>.<br />
3 http://etc-acc.eionet.eu.int/<strong>data</strong>bases/airview.html<br />
257<br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3
Figure 1<br />
The <strong>data</strong> flow from <strong>data</strong> supplier to AIRBASE and EBAS.<br />
The <strong>data</strong> supplier can also verify the imported <strong>data</strong> by checking on outliers, by visualising<br />
the <strong>data</strong> and by making reports. When the <strong>data</strong> supplier is finished, he sends<br />
the <strong>DEM</strong>-<strong>AQ</strong> to EEA’s Central <strong>Data</strong> Repository (CDR), which is a module of<br />
EEA’s Reportnet. For the countries which are not able to access the CDR the <strong>DEM</strong>-<br />
<strong>AQ</strong> can be uploaded to the ftp-server of the ETC/ACC. When the <strong>DEM</strong>-<strong>AQ</strong> contains<br />
only EMEP-<strong>data</strong>, the <strong>DEM</strong>-<strong>AQ</strong> is <strong>for</strong>warded to the ftp-server of EMEP.<br />
All <strong>DEM</strong>-<strong>AQ</strong> <strong>data</strong>bases in CDR are copied to the ETC/ACC ftp-server. The <strong>data</strong><br />
of the <strong>DEM</strong>-<strong>AQ</strong> <strong>data</strong>bases on the ftp-server are copied to special import tables in the<br />
AIRBASE <strong>data</strong>base be<strong>for</strong>e they are incorporated into the regular AIRBASE tables. The<br />
<strong>DEM</strong>-<strong>AQ</strong> <strong>data</strong>bases with EMEP <strong>data</strong> will export NASA Ames files <strong>for</strong> import into<br />
the EMEP-<strong>data</strong>base EBAS.<br />
258<br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3
3. <strong>DEM</strong>-<strong>AQ</strong><br />
The first version of the <strong>DEM</strong>-<strong>AQ</strong> was implemented in 1998. In <strong>2004</strong> <strong>DEM</strong>-<strong>AQ</strong> version<br />
7 is released. <strong>DEM</strong>-<strong>AQ</strong> is basically an MS-Access <strong>data</strong>base with a Visual Basic<br />
user interface. The software is freely downloadable 4 .<br />
When the <strong>data</strong> supplier receives the <strong>DEM</strong>-<strong>AQ</strong>, the first step is to check the meta<br />
in<strong>for</strong>mation and, if necessary, modify, add and delete the meta <strong>data</strong> (networks, stations<br />
and measurement configurations, see Figures 2 until 5).<br />
4 http:/etc-acc.eionet.eu.int/<strong>data</strong>bases/dem_install.html<br />
259<br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3
Figure 2: Network maintenance <strong>for</strong>m.<br />
Figure 3: Station maintenance <strong>for</strong>m, general tab.<br />
260<br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3
Figure 4: Station maintenance <strong>for</strong>m, classification tab.<br />
Figure 5: Measurement configuration maintenance <strong>for</strong>m.<br />
261<br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3
Figure 6: Import of <strong>data</strong> files.<br />
The next step is to select station - component combinations <strong>for</strong> which air quality<br />
<strong>data</strong> is to be exchanged, including subsequent linking of these combinations to air<br />
quality <strong>data</strong> files and import of these files with extensive error checking (Figure 6).<br />
Then the <strong>data</strong> supplier can check the imported <strong>data</strong> on extreme values (outliers) and<br />
control the <strong>data</strong> by visualisation (Figures 7 and 8).<br />
The <strong>data</strong> supplier can also generate reports and XML dumps of the contents of the<br />
<strong>DEM</strong>-<strong>AQ</strong>. He can also generate Nasa Ames files <strong>for</strong> incorporation in the EMEP <strong>data</strong>bases.<br />
262<br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3
Figure 7: Histogram imported <strong>data</strong>.<br />
Figure 8: Line graph imported <strong>data</strong>.<br />
263<br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3
The last step is transmitting the <strong>DEM</strong>-<strong>AQ</strong> <strong>data</strong>base through Internet (HTTP POST<br />
or ftp) <strong>for</strong> incorporation in the EEA’s Central <strong>Data</strong> Repository (CDR), EEA -<br />
ETC/ACC <strong>data</strong>base AIRBASE and the EMEP <strong>data</strong>base EBAS.<br />
Last year ETC/ACC received air quality <strong>data</strong> from 30 EEA-member states. Almost<br />
all countries have sent the <strong>DEM</strong>-<strong>AQ</strong>, a few countries delivered ISO-files or<br />
<strong>DEM</strong>-files.<br />
Be<strong>for</strong>e uploading AIRBASE ETC/ACC per<strong>for</strong>ms some quality checks and sent the<br />
results to the <strong>data</strong> suppliers <strong>for</strong> feedback. When the feedback is processed, all statistics<br />
and exceedances are calculated from the supplied raw <strong>data</strong> and loaded in<br />
AIRBASE.<br />
AIRBASE can be accessed interactively via an Web based Java application<br />
(AIRVIEW 5 ). Also each year XML dumps of the complete AIRBASE are available on<br />
the ETC/ACC Website 6 .<br />
4. Concluding remarks<br />
In this contribution a simple Access application is presented. With this tool it is possible<br />
to exchange air quality in<strong>for</strong>mation, both meta in<strong>for</strong>mation on networks, stations<br />
and measurement configurations and air quality measurement <strong>data</strong> in a harmonized<br />
and consistent way. Use of the <strong>DEM</strong>-<strong>AQ</strong> improves the quality considerably by<br />
the checks on <strong>for</strong>mat errors, outliers and obligatory meta <strong>data</strong> and the visualisation<br />
and report facilities. The <strong>DEM</strong>-<strong>AQ</strong> also stimulates the harmonisation of air quality<br />
<strong>data</strong> flows by the possibility to combine different <strong>data</strong> flows in one <strong>DEM</strong>-<strong>AQ</strong>. Harmonisation<br />
of obligatory air quality <strong>data</strong>flows (EoI, EMEP, Euro<strong>Air</strong>net), but also<br />
delivery of OSPARCOM <strong>data</strong> is possible with the <strong>DEM</strong>-<strong>AQ</strong>. Also harmonisation of<br />
meta <strong>data</strong> is improved: all meta <strong>data</strong> is centrally managed by <strong>Air</strong>Base. The <strong>data</strong> supplier<br />
can check the meta <strong>data</strong> in the preloaded <strong>DEM</strong>-<strong>AQ</strong>. And finally the <strong>DEM</strong>-<strong>AQ</strong><br />
has automatic upload facilities to EEA-CDR and the ETC/ACC ftp-server.<br />
Bibliography<br />
Mol, W.J.A. (2003): Coupling of <strong>Air</strong>Base with EEA Map Service Tool, in: The In<strong>for</strong>mation<br />
Society and Enlargement of the European Union, 17 th International<br />
5 http://etc-acc.eionet.eu.int/<strong>data</strong>bases/airview.html<br />
6 http://etc-acc.eionet.eu.int/<strong>data</strong>bases/<strong>Air</strong>BaseXML.html<br />
264<br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3
Conference In<strong>for</strong>matics <strong>for</strong> Environmental Protection Cottbus 2003, A.<br />
Gnauck, R. Heinrich (Eds.), Marburg, 2003, pp. 342-392<br />
265<br />
Copyright © Editions du Tricorne <strong>2004</strong>. ISBN: 28 29 30 275-3