4 years ago

Automatic Extraction of Examples for Word Sense Disambiguation

Automatic Extraction of Examples for Word Sense Disambiguation


CHAPTER 3. COMPARABILITY FOR WSD SYSTEMS 33 where sense inventories and considerably different levels of frequencies for a given word and/or sense are assembled) turns out to be pointless if an attempt to compare the systems’ results is planned further on in the process. Furthermore, the number of words that is used for testing will be as well significantly impor- tant to the results a system can accomplish. As Preiss (2006) reports Bruce and Wiebe (1994) used only a single word (interest) for the evaluation of their system, while Stevenson (1999) evaluated on all of the labeled words in the SemCor corpus (see Section 2.3.2). However, it is not only the number of test words that is used but as well their type - differ- ent parts of speech, degree of ambiguity, the degree of clarity between the distinctions amongst the senses of a word, the degree of sense granularity (fine sense distinctions vs. coarse sense distinctions), the frequency of their appearance in a metonymic or a metaphoric way, the inter- annotator agreement (IAA) (IAA is the percentage of times that the annotators chose the same sense for the targeted instance. In some cases the latter can have worse accuracy than ran- domly picking a sense - (Ahlswede, 1995) reports variations between 63.3% and 90.2% agreement among annotators) and so on and so forth. This more than complex constellation greatly serves to demonstrate the vivid need for what Gale et al. (1992) have proved to be an inseparable part of the evaluation of word sense dis- ambiguation systems - upper and lower bound for evaluation. The upper bound represents the highest possible accuracy that a system could have. Since it is unrealistic to expect that an au- tomated method is better in distinguishing between senses than a human is, the upper bound is set to be the agreement among the human annotators. The lower bound (also called baseline) on the other hand represents the performance of a very basic approach that has proved to yield relatively good results. It serves to delineate the lowest result that a system should have, since such a baseline result is already easily gained. 3.2 Most Frequently Used Baselines In the following section we discuss the most often used approaches to gain a comparable base- line for a WSD system. Unfortunately it is not always the case that they provide the needed information for a good comparison between WSD systems. Thus in Chapter 4 we will present another possibility for the evaluation of word sense disambiguation approaches. 3.2.1 The Lesk Algorithm The Lesk algorithm (Lesk, 1986), which we presented in Section 2.1.1, is one of the baselines usually used in WSD systems. However, the application of the original algorithm is in most cases more than is actually needed and therefore various simplifications or other modifications of it are normally taken into account e.g. variations of the context window to which the algorithm is applied, identical word forms that occur in the same definition are discarded, stemming or case

CHAPTER 3. COMPARABILITY FOR WSD SYSTEMS 34 corrections are left aside, usage of phrase filtering, etc. As reported by Mihalcea (2007) the Lesk algorithm reaches accuracy of 50-70% on a set of ambiguous word pairs, which were manually annotated (Oxford Advanced Learner’s Dictionary was used as sense inventory). 3.2.2 Most Frequent Sense The MFS (Gale et al., 1992) is even more straightforward than the Lesk algorithm. According to the statistics of the frequencies of targeted words, which are normally drawn from different corpora, the instances are labeled with the most frequent of the senses of the word that is dis- ambiguated. Another option for gaining the frequencies of the senses in the corpus on which it has been worked on is by performing domain analysis as in (Koeling et al., 2005). Extracting the frequen- cies of the current corpus, however, triggers in most cases the ”the chicken or the egg” problem since this already presupposes sense tagging of the text and thus an existing WSD system on the first place. The MFS approach has a relatively good performance. It is based on (Zipf, 1945) - the Zipfian distribution (a specific sense has a dominant frequency of occurrence, whereas all remaining senses of the target word share a considerable decrease in the frequency of occurrence) and reaches accuracy of about 55.2%. 3.2.3 Random Choice The random choice baseline assumes equal weights for all senses of the target word in order to choose one of them. Usually, however, the choice of the sense is not completely random but restrained by the POS tag of the targeted sense if such is provided beforehand. If not, all possi- bilities are considered equal.

A Machine Learning Approach for Automatic Road Extraction - asprs
Selective Sampling for Example-based Word Sense Disambiguation
Word sense disambiguation with pattern learning and automatic ...
Word Sense Disambiguation Using Automatically Acquired Verbal ...
Using Machine Learning Algorithms for Word Sense Disambiguation ...
Word Sense Disambiguation The problem of WSD - PEOPLE
Performance Metrics for Word Sense Disambiguation
Word Sense Disambiguation - cs547pa1
Word Sense Disambiguation
Word Sense Disambiguation Using Selectional Restriction -
Using Lexicon Definitions and Internet to Disambiguate Word Senses
MRD-based Word Sense Disambiguation - the Association for ...
word sense disambiguation and recognizing textual entailment with ...
A Comparative Evaluation of Word Sense Disambiguation Algorithms
KU: Word Sense Disambiguation by Substitution - Deniz Yuret's ...
Semi-supervised Word Sense Disambiguation ... - ResearchGate
Using unsupervised word sense disambiguation to ... - INESC-ID
Using Meaning Aspects for Word Sense Disambiguation
Word Sense Disambiguation: An Empirical Survey - International ...
Word Sense Disambiguation is Fundamentally Multidimensional
Word-Sense Disambiguation for Machine Translation
Unsupervised learning of word sense disambiguation rules ... - CLAIR
Word Sense Disambiguation Using Association Rules: A Survey
Similarity-based Word Sense Disambiguation
Word Sense Disambiguation with Pictures - CiteSeerX
Ch7. word sense disambiguation
Word Sense Disambiguation with Pictures - CLAIR