A common critique of measuring Open Data is that metrics can be misleading: the most commonly used metric is the number of datasets in a catalogue. While this is a good indicator of a rich Open Data ecosystem, it is by no means guaranteed that counts of datasets are comparable across portals or even time. The number may be artificially inflated because of different units of analysis or, more prosaic, different ways to count. As a rule of thumb, the more meaningful a construct appears, the harder it is to measure. The exact definition here and in other places often remains vague, although there have been attempts to define specific metrics. 84 7.4 OpenDataMonitor Figure 7 illustrates OpenDataMonitor’s approach to automated metrics, and the tension between what is measurable and what is difficult to measure in an automated way. OpenDataMonitor, 85 a European Commission-funded project which ran from 2013-15, was the first of its kind to automatically harvest and harmonise, measure and monitor, analyse and visualise metadata from available datasets and catalogues at a regional, country and pan-European level. Figure 7: Mapping Open Data concepts to dimensions of availability and measurability 86 84 European Data Portal, 2016, Open Data Maturity in Europe 2016: Insights into the European state of play 85 Open Data Monitor 86 OpenDataMonitor, 2013, Best Practice Visualisation, Dashboard and Key Figures Report 58
The intended audience of the project and its tools were portal owners, policymakers, journalists and citizens. It therefore has a broad appeal and provides helpful lessons in applying metrics to Open Data. By measuring Open Datasets harvested from portals across Europe, various aspects were studied with regards to quality and quantity. Quantity measures explored by OpenDataMonitor included: total number of datasets total number of distributions total number of catalogues total distribution size number of unique publishers Quality measures explored by OpenDataMonitor included: Percentage of datasets with an open licence Percentage of machine-readable datasets Percentage of datasets with complete metadata Percentage of available datasets Percentage of datasets in an open format Figures 8 and 9 are reproduced from the OpenDataMonitor website 87 , and detail the quantity and quality automated metrics derived from data catalogues across Europe. Figure 8: Quantity metrics from OpenDataMonitor 87 OpenDataMonitor, Methodology of OpenDataMonitor 59
The Open Data Institute prepared th
Table of Contents List of figures .
List of figures Figure 1: 25 years