Proceedings of the 12th European Conference on Knowledge ...

Taha Osman, Dhavalkumar Thakker and Matt Nathan
inferencing. In this contribution, we focus on <strong>the</strong> technologies that are directly related to knowledge
management, namely information extraction (IE), ontology engineering, and <strong>the</strong> knowledgebase.
Next we document our experience in exploiting and integrating <strong>the</strong> above technologies to build an
intelligent browsing engine for PA Images, in view to contributing to <strong>the</strong> methodology <strong>of</strong> building
semantic-driven knowledge management frameworks for this class <strong>of</strong> applications.
4.1 Information extraction
Image
captions
GATE-based IE System
Gazetteer (known entities)
JAPE Grammar (context rules)
Disambiguation/Summarisation
Annotated Image
Captions
What to store
What to extract
Confirmation
Entities <strong>of</strong> Interest Learned Facts
PA Knowledgebase
PA Images
Ontology
Schema
Data
PA Images view
Linked
Data Cloud
Figure 1: Semantic Annotation System
It is fundamental to semantic retrieval systems that <strong>the</strong>re is a mechanism to annotate images with
descriptive metadata (entities) representing key concepts and relations <strong>of</strong> <strong>the</strong> investigated application
domain. In our framework as illustrated in Figure 1 below, <strong>the</strong> metadata generation process is
handled by <strong>the</strong> GATE based text mining system (Cunningham, 2002) that takes advantage <strong>of</strong> <strong>the</strong> rich,
domain-specific PA Images’ knowledgebase. PA Images’ knowledgebase consists <strong>of</strong> ontologies
(schema) and data operating on <strong>the</strong> schema.
4.1.1 Entity recognition and disambiguation
The company receives images from a number <strong>of</strong> contracted photographers and photo agencies with
minimal metadata, such as a short description <strong>of</strong> <strong>the</strong> main event in <strong>the</strong> image (headline) and a more
elaborate description <strong>of</strong> <strong>the</strong> image with detailed event information (caption). The text mining system
operates on headline and caption to extract entities <strong>of</strong> interest.
The IE system utilises GATE for text mining and contains three components: gazetteers, JAPE
grammar and a disambiguation module. The gazetteers are <strong>the</strong> list <strong>of</strong> known entities that <strong>the</strong> system
utilises during <strong>the</strong> initialization process. The ontology influences <strong>the</strong> decision on what information
needs to be stored in <strong>the</strong>se gazetteers. The JAPE grammar rules allow detecting additional entities
and at <strong>the</strong> same time confirming <strong>the</strong> entities detected by <strong>the</strong> gazetteers.
The disambiguation module deals with any disambiguation generated by previous components. Here,
<strong>the</strong> disambiguation refers to <strong>the</strong> cases where <strong>the</strong> same piece <strong>of</strong> text is ei<strong>the</strong>r given two class labels
(e.g. “Liverpool” as City and Club) or where two or more entity identifier is assigned to <strong>the</strong> same piece
<strong>of</strong> text (e.g. “Premier League” as “id:Premier_Legue_Darts” and “id: Premier_Legue_Football”). We
made a design decision whereby <strong>the</strong> IE system contains limited knowledge relating to <strong>the</strong> entities as
<strong>the</strong> gazetteers contain only alternate labels and classification <strong>of</strong> <strong>the</strong> entities. Moreover, <strong>the</strong>
knowledgebase also contains relationships between <strong>the</strong>se entities. Therefore, <strong>the</strong> knowledgebase
plays a crucial role in resolving disambiguation as it has more embedded intelligence compared to <strong>the</strong>
IE system.
The information extracted using <strong>the</strong> IE system generally has a confidence rating to indicate <strong>the</strong>
confidence <strong>of</strong> <strong>the</strong> system in <strong>the</strong> process. For example, higher level <strong>of</strong> confidence is assigned to
extracted entities where <strong>the</strong> entities are already known to <strong>the</strong> gazetteers and <strong>the</strong>re is sufficient context
in <strong>the</strong> text to confirm <strong>the</strong> validity <strong>of</strong> <strong>the</strong> extraction. The summarization module filters <strong>the</strong> results for
740