NUI Galway – UL Alliance First Annual ENGINEERING AND - ARAN ...

Curated Entities for Enterprise
Umair ul Hassan, Edward Curry, Seán O’Riain
Digital Enterprise Research Institute
National University of Ireland, Galway
umair.ul.hassan@deri.org, ed.curry@deri.org, sean.oriain@deri.org
Abstract
We propose an entity consolidation system with user
collaboration. First source data is converted into
entity-attribute-value data model. Then system finds
equivalence relationships at entity, attribute, and value
levels. The confidence of relationships and data
conflicts are kept as intermediate results. Users provide
feedback on result iteratively. Finally corrected data
with lineage and provenance information is updated in
curated database of entities.
1. Introduction
The amount of data generated and stored in
organizations is increasing with more automation of
business processes. Often this data relates to specific
entities of business interest like people, products, and
customers. Users collect, integrate and standardize this
data for analysis. Teams of analysts and skilled IT staff
spend significant amount of time and effort to bring this
all at one place.
2. Problem Statement
Integration of data from disparate sources generates
uncertain results [1]. For example if an analyst
integrates data about iPod from two sources, following
types of uncertainty can occur for price of iPod
Absence: no price
Conflicts: price is 150€ and also 160€
Vagueness: price is given as High
Non-specificity: price is between 150€ and 160€
3. Proposal
We propose to develop an entity consolidation
system, which supports iterative cleaning of uncertain
data with user feedback [2]. Figure 1 illustrates major
process flow steps of our prototype.
Figure 1: Process flow of entity consolidation with management of
uncertain data using iterative user feedback
115
3.1. Entity Consolidation
The process starts by converting source data in
common entity-attribute-value format [3]. Followed by
three associated tasks; mapping of schema attributes
between sources, comparing individual entities for
equivalence, and merging values of attributes for same
entities.
3.2. Uncertain Data
Automated entity consolidation generates results
with confidence scores for equivalence between
entities. Conflicts of data values also exist between
matched entities. All this information is stored in
temporary database for resolution.
3.3. User Feedback
User provides feedback on uncertain data in two
forms, either by validating possible choices or
providing generic rules for repairs. Having people with
domain expertise collaborate to improve quality of
integration result adds value to overall process. This is
similar to curation process of reference works and
dictionaries by domain experts [4].
3.4. Provenance
Provenance information about data source, entities,
and user feedback is stored for tracking lineage of data.
This information serves as indicator of trust for entity
database consumers, which can be further utilized to
support data cleaning tasks automatically.
4. References
[1] M. Magnani and D. Montesi, “A Survey on
Uncertainty Management in Data Integration,” Journal
of Data and Information Quality, vol. 2, Jul. 2010, p.
33.
[2] M.J. Franklin, “Dataspaces: Progress and
Prospects,” Dataspace: The Final Frontier, A.P.
Sexton, ed., Berlin, Heidelberg: Springer Berlin
Heidelberg, 2009, pp. 1-3.
[3] P.M. Nadkarni, L. Marenco, R. Chen, E. Skoufos,
G. Shepherd, and P. Miller, “Organization of
Heterogeneous Scientific Data Using the EAV / CR
Representation,” Journal of the American Medical
Informatics Association, vol. 6, 1999, pp. 478-493.
[4] P. Buneman, J. Cheney, W.-C. Tan, and S.
Vansummeren, “Curated databases,” Proceedings of the
twenty-seventh ACM SIGMOD-SIGACT-SIGART
symposium on Principles of database systems - PODS
’08, New York, New York, USA: ACM Press, 2008.