96 • 2011 - Hamburger Zentrum für Sprachkorpora - Universität ...

More documents

Recommendations

Info

Multilingual Resources and Multilingual Applications - Regular Papers Our approach is rule-based and uses the open geographical database OpenStreetMap1 as well as country-specific rules and patterns. It is robust and easily extensible to further languages. 2. Related Work Most work concerned with the segmentation of location information is based on statistical techniques (Borkar et al., 2001; Agichtein & Ganti, 2004; Christen & Belacic, 2005; Christen et. Al, 2002; Peng & McCallum, 2003; Marques & Gon Calves, 2004; Cortez & De Moura, 2010). However, as the experiments by Borkar et al. (2001) show, these methods drastically decrease in performance once confronted with a mixture of location strings from different countries. While an experiment on uniformly formatted address data from the U.S. reaches 99.6% accuracy, performance drops to 88.9% when trained and tested on addresses from mixed countries2 . There are only few published approaches on rule-based systems (Appelt et al, 1992; Riloff, 1993). Rule-based systems are generally thought to be less robust to noise, harder to adapt to other languages and thus considered suitable mainly for small domains. However a comparison of a rule based and statistical system (Borkar et al., 2001) showed that rules can compete with statistical approaches especially on inhomogeneous data. Given the fact, that huge geographical databases have become available in recent years, high-quality rulebased systems can be developed for large unrestricted domains with relatively little effort and easily be extended to more languages by adding more databases for the relevant countries. 3. Location information Segmentation Figure 1 shows the general architecture of our system. In a preprocessing step the location information string is tokenised and normalised according to country-specific normalisation patterns (e.g. str becomes straße). Initial grouping is done if applicable, i.e. if indications for grouping already exist. These steps are necessary for a 1 http://www.openstreetmap.org 2 The accuracy measure used in this article is an overall measure of all element-wise measurements for the address elements under consideration and similar to the labelled recall measure used in Section 4. 40 later OpenStreetMap query because abbreviations or partial street names cannot be found in the database. Figure 1: The system architecture In the succeeding tagging step all identifiable geographical names are tagged by querying OpenStreetMap, allowing tagging ambiguities. Countryspecific string patterns aid the tagging of elements containing numbers. One of the difficulties within this step is that address elements often consist of more than one token. The challenge lies in querying for Oxford Street and not erroneously tagging Oxford as the place name and leaving Street untagged. This is achieved by a longest match policy. However, all match information is preserved. Multiple queries are used to account for diacritical variation as in umlauts in German (e.g. ü) and parentheses as parts of official geographical names (as in Frankfurt(Main)/Frankfurt Main) and other non alpha-numercial marks such as hyphens. As a result of this step, the elements are tagged with OpenStreetMap (OSM) internal types and not yet the address element types we are looking for. OSM types are often ambiguous. The string Potsdam, Brandenburg is tagged as Potsdam (county/city/hamlet) Brandenburg (town, state) , for instance3 . 3 For a human reader familiar with the location it is clear, that this denotes the city of Potsdam within the state of Brandenburg, even though there is also a city called Brandenburg in the state of Brandenburg, for instance, or a hamlet called Potsdam in the state of Schleswig-Holstein.
Multilingual Resources and Multilingual Applications - Regular Papers The following step maps OSM types to address elements. In the OpenStreetMap project, every country- specific subproject, e.g. the Japanese or the German OSM project, has its own guidelines about how to tag locations according to their administrative unit status (as being a city, town or hamlet4 ). Therefore we use country specific mappings from OSM internal types to one or more of the desired target address element types we define. The enrichment step provides rules for labelling address elements which have not been attributed a tag by a previous step because they were not found in the knowledge base due to spelling errors, for instance. The completion rules are of the following form: (type1 , type2 , . . . , typen ) − − > targetAddressElement If for each typex, x = 1 .. n, for the token at index x, the respective type can be found in the list of possible types, the tokens in the sequence are grouped and labelled with the type targetAddressElement. Examples for language specific completion token types are found in Table 3. A token tagged with one of these affix types indicates a (possibly still unlabelled) preceding/following location name and the token group is labelled appropriately including the marker token. compl. type examples description town_suf ku Suffix marking a station_suf Station, Ekimae, Meieki town/ward (Japan) Word marking a train station (Japan) village_suf mura, son Suffix marking a village (Japan) city_dist_pref Aza, Koaza Prefix usually preceding a city district or sub- district (Japan) street_suf Avenue, Suffix marking a street Road name (Australia) state_pref Freistaat Prefix marking a state name (Germany) Table 3: Completion types Some examples of completion rules are listed in Table 4. The left hand side of the rules specifies the token type pattern, the right hand side defines the target address element. An @ means that the token at the respective 4 A hamlet is a small town or village. position must not have other possible types than the specified one. completion rule matching example (city_prefix,city) --> city Hansestadt (orientation_prefix,other, street_suffix) --> street_name Hamburg Lower Geoge Street (instead of George Street) (orientation_prefix,city) East Launcheston (contains_street_suffix) --> Ratausstraße street_name (instead of Rathausstraße (city,loc_suffix) --> city_district Berlin Mitte (state_prefix,state) --> state Freistaat Bayern (@city,@city) --> city Munich (München) (street_number,street_number_ext) --> street_number 34a (street_number,sep_last_alphanum) --> street_number 34 - 36 Table 4: Example completion rules The final disambiguation step provides rules which decide which of the attributed types for each element is selected. In the aforementioned example, Brandenburg would thus be tagged a state and not a city. The disambiguation rules take the form (leftNeighbourType, currentType, rightNeighbourType) where currentType is the target address element type of the token group under consideration. Either rightNeighbourType or leftNeighbourType may be empty (i.e. any type is allowed). If such a rule can be applied, the token group under consideration will be labelled with currentType. 4. Experiments 4.1. Data We conducted our experiments using two different datasets. The first dataset was collected from the Internet, the second corpus was a company internal database. Eleven external annotators collected variations of location information data from the Internet and annotated them according to the annotation guidelines given in Wolf (<strong>2011</strong>). They collected 154 strings for German, 35 of which were used for development and the rest for testing. For Australia they collected 143 strings, 41
Page 1: ARBEITEN ZUR MEHRSPRACHIGKEIT WORKI
Page 4: Collaborative Research Center: Mult
Page 7 and 8: Call for Papers The Conference of t
Page 9 and 10: Hybrid Machine Translation for Germ
Page 11: IV System Presentations New and Fut
Page 14 and 15: Multilingual Resources and Multilin
Page 96 and 97:
Multilingual Resources and Multilin
Page 98 and 99:
Page 100 and 101:
Page 102 and 103:
Page 104 and 105:
Page 106 and 107:
Page 108 and 109:
Page 110 and 111:
Page 112 and 113:
Page 114 and 115:
Page 116 and 117:
Page 118 and 119:
Page 120 and 121:
Page 122 and 123:
Page 124 and 125:
Page 126 and 127:
Page 128 and 129:
Page 130 and 131:
Page 132 and 133:
Page 134 and 135:
Page 136 and 137:
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Page 154 and 155:
Page 156 and 157:
Page 158 and 159:
Page 160 and 161:
Page 162 and 163:
Page 164 and 165:
Page 166 and 167:
Page 168 and 169:
Page 170 and 171:
Page 172 and 173:
Page 174 and 175:
Page 176 and 177:
Page 178 and 179:
Page 180 and 181:
Page 182 and 183:
Page 184 and 185:
Page 186 and 187:
Page 188 and 189:
Page 190 and 191:
Page 192 and 193:
Page 194 and 195:
Page 196 and 197:
Page 198 and 199:
Page 200 and 201:
Page 202 and 203:
Page 204 and 205:
Page 206 and 207:
Page 208 and 209:
Page 210 and 211:
Page 212 and 213:
Page 214 and 215:
Page 216 and 217:
Page 218 and 219:
Page 220 and 221:
Page 222 and 223:
Page 224 and 225:
Page 226 and 227:
Page 228 and 229:
Page 230 and 231:
Page 232 and 233:
Page 234 and 235:
Page 236 and 237:
Page 238 and 239:
Page 240 and 241:
Page 242 and 243:
Page 244 and 245:
Page 246 and 247:
Page 248 and 249:
Page 250 and 251:
Page 252 and 253:
Page 254 and 255:
Page 256 and 257:
Page 258 and 259:
Page 260 and 261:
Page 262 and 263:
Page 264 and 265:
Page 266 and 267:
Page 268 and 269:
Page 270 and 271:
Page 272 and 273:
WORKING PAPERS IN MULTILINGUALISM
Page 274:
74. Ludger Zeevaert: Variation und
show all

96 • 2011 - Hamburger Zentrum für Sprachkorpora - Universität ...

Create successful ePaper yourself

Delete template?

Save as template?