1 year ago

European Commission Project team DISCLAIMER


The next iteration of

The next iteration of this report will explore further efforts to develop automated metrics for measuring the use and impact of open data and their feasibility (see Next Steps). In addition to discovering examples of re-use, portal owners face the challenge of trying to understand their re-users. Open Data Maturity in Europe 2016 found that ‘only 48% of the countries with a national Open Data portal have a basic overview of the typical profile of their visitors.’ Our research supports this finding; most portal owners we spoke to explained why this was both important and difficult. One respondent told us that they ‘would really like to capture the needs of people… [to] mov[e] from metrics that capture supply towards demand.’ Another explained that, ‘you need knowledge of your users’, so they ‘encouraged registration for accessing data’ but did not want it to be mandatory. The majority of portal owners told us they also wanted to measure: which datasets were accessed most frequently which combinations of datasets were accessed together which datasets were not updated the percentage of datasets that had open licences the percentage of datasets that were machine readable how many resources were downloaded how many geodata formats were downloaded the number of linked datasets the portal had 7.10 Overcoming challenges in automating metrics Our research has shown attempts to measure and benchmark Open Data portals, related efforts by the European Data Portal and projects such as OpenDataMonitor to automate metrics, what portal owners can measure and what they would like to be able to measure. Challenges remain to capture metrics, which must be addressed before automation can be considered. These include: the potential gap between what metadata represents and how it is measured the need to quantify complex constructs the often missing information in the first place the potential for tensions between audiences and the issue of gaming metrics functionality and capability from platform providers the high cost in time and resources for building automation uncertainty about how future technological developments might affect automation Despite all of this, we are seeing promising efforts from portal owners and a desire to measure much more than they are currently capable of. Further work is needed to understand what portal owners across other countries, cities and regions are doing to set and automate metrics for measuring their success, and what they would like to measure, overlapped with a limitation in portal functionality. 68

Existing web analytics tools like the open-source analytics platform, Piwik 103 , and Google Analytics, help to capture how frequently data portals are visited, for how long and whether users continue to return to the portal as a useful source of data. Using web analytics to understand a site’s user base is already common-place for content editors and web publishers. Similarly easy to use tools that automate measures of data quality (in what format it is published, how it is updated and licensed) and assess metadata quality will continue to evolve. Different metrics for Open Data serve different audiences. Some help data publishers measure themselves against other data publishers in comparable organisations and sectors. Others help to guide end-users of data to data that is already widely used, or of a particular quality. Measures of metadata quality help portal owners assess the extent to which data accessible via the portal is discoverable and useful, and whether data publishers are complying with metadata standards. While work on automated metrics continues to evolve towards widespread adoption, the varied audiences for metrics is worth keeping in mind. This is reflected in the recommendations for this section, which highlight the needs of both publishers and users: Choose metrics that help to benchmark data publisher performance, but do not rely on one metric e.g. quantity. Combine quantity metrics with data quality and engagement measures Choose metrics that help potential data users find data that is suitable for them to use. Evaluate whether the metrics chosen are meaningful or potentially misleading. Continuing to invest in automation of metadata quality assessments, like the MQA dashboard developed by the EDP, provide a quick shorthand for portal owners to identify and respond to quality issues. Making widely accepted metadata standards, like the Data Catalogue Vocabulary (DCAT) the basis for measuring quality will help to accelerate uptake, and create a level playing field for data publishers and users. Metrics need to be integrated into the wider context so that they lead to decision-making and policy formulation. Open Data is an agent for change, and when an organisation designs a theory of change, the golden thread that runs through this is a well thought out monitoring and evaluation plan. Understanding the desired impact and setting metrics to measure this, as well as plans for sustainability, underpin such a strategy. 103 PIWIK 69

THE QUERY PROJECT - European Commission - Europa
Co-operative Research projects for SMEs - European Commission ...
August 2004 - European Commission - Europa
The march towards equality - European Commission - Europa
GOOD PRACTICE GUIDE - European Commission - Europa
Roberto Cencioni European Commission DG ... - LT-Innovate
mapping of the european research infrastructure landscape
ERRIN booklet - European Commission - Europa
European Commission policy on open access, Celina - UNICA
Research for a Secure Europe - European Commission - Europa
O.8 European Commission Funded Projects - Eurochambres Academy
Learning with eTwinning - European Commission - Europa
ERA-NET - European Commission - Europa
ICT for Societal Challenges - European Commission - Europa
Synthesis Report - European Commission - Europa
Sara Sköld – Clean Shipping Project - European Shortsea Network
Sub group on Private Equity funds - European Commission - Europa
Jean-Yves ROGER European Commission DG INFSO IST ... - Ercim
Research for a Secure Europe - European Commission - Europa
European Commission Requirements for Projects - IWR
SMEs and Research - European Commission - Europa
LIFE+ Nature & Biodiversity Projects 2011 - European Commission
European Commission initiatives on CCS -update from the research ...
Research regionally develop globally - European Commission
business aircraft owners and operators in europe - European ...