Ch7. word sense disambiguation

More documents

Recommendations

Info

EE669 Natural Language Processing Fall 2001EE669 Notes 10Unsupervised Disambiguation• It may be useful to disambiguate among differentword senses in cases where there are no availablelexical resources.– in a specialized domain (e.g., linguistics)– could be quite important for information retrieval in adomain• Of course, it is impossible to do sense tagging in asituation where there is no labeled data; however,it is possible to carry out sense discrimination in acompletely unsupervised manner.Unsupervised Disambiguation• Without supporting tools such as dictionaries andthesauri and in the absence of labeled text, we cansimply cluster the contexts of an ambiguous wordinto a number of groups and discriminate betweenthese groups without labeling them.• Context-group discrimination (Schutze, 1998):– Clusters uses of an ambiguous word with no additionalknowledge.– For an ambiguous word w with senses s 1 , …, s k , …, s K ,estimate the conditional probability of each word v joccurring in w’s context being used with sense s k ,P(v j |s k ).Fall 2001 EE669: Natural Language Processing 37Fall 2001 EE669: Natural Language Processing 38Schutze (1998)• The probabilistic model is the same Bayesian model as theone used by Gale et al.’s Bayes classifier, except that eachP(v j |s k ) is estimated using the EM algorithm.– Start with a random initialization of the parameters of P(v j |s k ).– Compute for each context c i of w, the probability P(c j |s k ) generatedby s k ..– Use this preliminary categorization of contexts as our training dataand then re-estimate P(v j |s k ) to maximize the likelihood of the datagiven the model.– EM is guaranteed to increase the log likelihood of the model giventhe data at each step; therefore, the algorithm stops when thelikelihood does not increase significantly.Fall 2001 EE669: Natural Language Processing 39Schutze (1998)• Once model parameters are estimated, we candisambiguate contexts w by computing the probability ofeach of the senses based on the words v j occurring incontext. Schutze (1998) uses the Naïve Bayes decisionrule:– Decide s’ if s’=argmax sk [log P(s k )+Σ vj in c log P(v j |s k )]• The granularity of senses of a word can be chosen byrunning the algorithm over a range of values.– The larger the number of senses the better it will be able to explainthe data.– Relative increase in likelihood may help to distinguish importantsenses from random variations.– Could make # of senses dependent on the amount of training data.– Can get finer grained distinctions than in supervised approaches.• Works better for topic-dependent senses than topicindependentones.Fall 2001 EE669: Natural Language Processing 40
EE669 Natural Language Processing Fall 2001EE669 Notes 11So What is a Word Sense Really?• It might seem reasonable to define word senses asthe mental representations of different wordmeanings.– Not much is known about mental representationsbecause it is hard to design experiments to get at whatthat is.– Humans can categorize word usage using introspection,but is that reasonable? Also agreement tends to be low.– Humans could label word senses using dictionarydefinitions, but this works best for skewed distributionswhere one sense is predominant. Also, definitions canoften be vague.– Words with the highest frequencies have the highestdisagreement rate, so selecting words based onfrequency would bias results.Fall 2001 EE669: Natural Language Processing 41So What is a Word Sense Really?• It may be that it is common for humans to have asimultaneous activation of different senses whencomprehending words in text or discourse (leading to highlevels of disagreement).• These coactivations may be cases of systematic polysemy,where lexico-semantic rules apply to the class of words tosystematically change or extend their meaning. Forexample, competition can refer to the act of X or the peopledoing X.• Propernouns also create problems, e.g., Brown, Army, etc.• Could consider only course-grained distinctions amongword senses (like those that show up across languages).Clustering approaches to word sense disambiguation adoptthis strategy.Fall 2001 EE669: Natural Language Processing 42Word Sense DisambiguationEvaluation• If the disambiguation task is embedded in a task liketranslation, then it is easy to evaluate in the context of thatapplication. This leads to application-oriented notions ofsense.• Direct evaluation of disambiguation accuracy is moredifficult in an application-independent sense. It would beeasier if there were standard evaluation sets (Sensevalproject is addressing this need).• There is a need for researchers to evaluate their algorithmson a representative sample of ambiguous words.Factors Influencing the Notion ofSense• The type of information used in disambiguation affects thenotion of sense used:– Co-occurrence (bag-of-words model): topical sense– Relational information (e.g., subject, object)– Other grammatical information (e.g., part-of-speech)– Collocations (one sense per collocation)– Discourse (one sense per discourse segment): How much context isneeded to determine sense?– Combinations of the above• Different types of information may be more useful fordifferent parts of speech (e.g., verb meaning is affected byits complements, but nouns are more affected by widercontext).Fall 2001 EE669: Natural Language Processing 43Fall 2001 EE669: Natural Language Processing 44
Page 4: EE669 Natural Language Processing F
Page 9: EE669 Natural Language Processing F

Ch7. word sense disambiguation

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?