96 • 2011 - Hamburger Zentrum für Sprachkorpora - Universität ...

Multilingual Resources and Multilingual Applications - Regular Papers
Tackling the Variation in International Location Information Data: An
Approach Using Open Semantic Databases
Janine Wolf 1 , Manfred Stede 2 , Michaela Atterer 1
1 Linguistic Search Solutions R&D GmbH, Rosenstraße 2, 10178 Berlin
2 Universität Potsdam, Karl-Liebknecht-Straße 24-25, 14476 Potsdam
E-mail: janine@wolf-velten.de, stede@uni-potsdam.de, michaela.atterer@lssrd.de
Abstract
International location information ranges from mere relational descriptions of places or buildings over semi-structured address-like
information up to fully structured postal address data. In order to be utilized, e.g. for associating events or people with
geographical information, these location descriptions have to be decomposed and the relevant semantic information units have to
be identified. However, they show a high amount of variation in order, occurrence and presentation of these semantic information
units. In this work we present a new approach of using a semantic database and a rule-based algorithm to tackle the variation in
such data and segment semi-structured location information strings into pre-defined elements. We show that our method is highly
suitable for data cleansing and classifying address data into countries, reaching an f-score of up to 97 for the segmentation task, an
f-score of 91 for the labelled segmentation task, and a success rate of 99% in the classification task.
Keywords: address parsing, OpenStreetMap, address segmentation, data cleansing
1. Introduction
Databases of international location information, as
maintained by most companies, often contain
incomplete address data, variation in the order of
elements, mixing of international conventions for
address formatting or even semi-translated address parts.
Moreover, the address data can be structured
insufficiently or erroneously according to the database
fields which makes the data unusable for further
classification, querying and data cleansing tasks.
Table 1 shows a number of possible variations of the
same German address.
address string problem description
Willy-Brandt Street 1, Berlin partial translations
#1 Willy-Brandt Street, Berlin 1000 non-standard format
Willy-Brand-Str. 1 incorrect spelling
Willy-Brandt-Str. 1, 1000 Berlin 20 politically outdated
Willy-Brandt-Str.1, Haus 1
presence of more
3.Et., Zi. 101
detailed information
In der Willy-Brandt-Str in Berlin incomplete, e.g.
extracted from free text
Table 1: Examples of variation in postal addresses based on
the German address Willy-Brandt-Str. 1, 10557 Berlin
Apart from this kind of variation we also face variation
in the description of location objects such as colloquial
variations as Big Apple for New York, historical
variations (Chemnitz/Karl-Marx-Stadt), transcription
variants (Peking/Beijing) or translation variants
(München/Munich).
International addresses create further variation in
address data as the typical Japanese address shown in
Table 2 exemplifies.
part of
description string
element type
11-1 street number (mixed information:
estate and building no.)
Kamitoba-hokotate-cho city district
Minami-ku ward of a city (town)
Kyoto city (here: also prefecture)
601-8501 postal code
Table 2: Address elements: Japanese postal address
example 11-1 Kamitoba-hokotate-cho,
Minami- ku, Kyoto 601-8501
All these variations pose major problems for data
warehousing, such as deduplication, record linkage and
identity matching.
In this work we propose a method which is highly
suitable for data cleansing. Tests on German, Australian
and Japanese data show that it is moreover suitable for
classifying address data into countries.
39