Designing a Machine Translation System for Canadian Weather ...

Designing a Weather Warning Translation System 26
Strong easterly winds with gusts up to 100 km/h are forecast to
develop in the Pincher Creek region this morning and then spread
eastward throughout the day. The strong winds will gradually
diminish this evening but will redevelop on Wednesday.
Fig. 8. An excerpt of a discussion as it appeared on EC’s public weather warning
website. We underlined mixed case phrases.
6.2 Truecasing
6.2.1 Principle
An example of the end result of the truecasing step is shown in Fig. 1, whose text
is reproduced in Fig. 8 for convenience, in which the tokens with a mixed case are
underlined. On the one hand, we observe that case restoration involves conceptually
simple rule-based transformations, like the capitalization of the first word of a
sentence or that of a weekday or month. We readily implemented these rules. On
the other hand, truecasing the numerous names of places and other named entities is
trickier. It actually consists of two problems: performing named entity recognition,
and then properly truecasing these named entities. Named entity recognition is
made difficult by the lack of case information.
For these reasons, and for the sake of simplicity, we chose a gazetteer-based
approach to truecasing. We started by compiling a list as exhaustive as possible of
all mixed case expressions susceptible to appear in Watt’s translations. We will
describe this preliminary step in detail in Section 6.2.2.
We compiled two lists, one for each output language, and made sure these lists
spelled each place name with its correct case. We added to these lists other capitalized
words (e.g. “Environment Canada”), extracted from sources described in
Section 6.2.2. An excerpt of the English entries is shown in Table 9.
With these lists at our disposal, it is then possible to scan the raw translations
in order to find these expressions in their raw form. Whenever a match is found,
the raw translation is replaced with the truecased expression. Each list is compiled
into a trie in order to speed up the matching step. We use a greedy algorithm that
scans the raw translation from left to right to find the longest matching list entry.
For instance, the raw “AN AREA NEAR METSO A CHOOT INDIAN RESERVE 23” will
only match Table 9’s “Metso A Choot Indian Reserve 23” entry, and not shorter
expressions.
It is noteworthy that our lists hold entries that contain diacritical marks (e.g.
“Metro Montréal” in Table 9). This allows a few final additions to the accenting
step of French text, described earlier (see Section 6.1). As for English text, this
single step is sufficient to restore the few diacritical marks necessary and no further
processing was deemed necessary.