MnrAq

Annex 4
Details for Section 1.5.1:
top-down uncertainty analysis
Organization of top-down uncertainty analysis
The top-down uncertainty section begins with a review of ongoing data accuracy efforts in which the
International Energy Agency (IEA) has participated, and data accuracy reports produced by the Energy
Information Administration (EIA), an energy statistics activity independent of IEA. We also summarize some
additional literature that helps to understand uncertainty in energy statistics. We then summarize four specific
sources of uncertainty in IEA data. With this information, we present our work to estimate possible sources
and quantities of uncertainties that may adjust reported statistics. Our quantification of potential adjustments
to fuel statistics distinguishes sources with the greatest impact on fuel statistic uncertainty – primary (or firstorder)
sources – from secondary and tertiary sources.
Ongoing data quality efforts related to uncertainty in fuel sales
The Joint Organisations Data Initiative (JODI) has worked since 2001 to produce a database to provide more
transparency in oil market data. The data effort includes “the collection of monthly oil statistics from each
organisation’s member countries by means of a harmonised questionnaire on 42 key oil data points”. More
than 90 countries/economies, members of the six pioneer organizations (APEC, EUROSTAT, IEA, OLADE,
OPEC and UNSD) participate in JODI-Oil, representing around 90% of global oil supply and demand. Among
the important work this group is performing, JODI is engaged in data quality assessment. That work appears
to be focused on uncertainties related to several elements, including:
1 data validation;
2 intercomparison with other energy statistics;
3 data collection; and
4 metadata.
While much of the current work seems to be engaging knowledge transfer through workshops and training
exchanges, the group has produced two approaches to characterizing data participation and content quality.
These are available in what JODI reports as smiley-face assessments, produced every six months since 2012
(Barcelona, 2012). Currently, these are qualitative assessments only, and could not be used in the quantitative
uncertainty analysis required for this work.
Review of EIA accuracy analyses (estimation of percentage error)
EIA resources were evaluated for a) similarity to IEA statistics, and b) complementary data quality investigations.
A discussion of the comparison of EIA similarity for fuel oil statistics was provided in the QA/QC section
under Section 1.4. Here we discuss the EIA reports on data accuracy as independent and indirect evidence of
sources and magnitude of uncertainty in top-down fuel consumption statistics.