Collocation Collocations & colligations Defining ... - Indiana University

More documents

Recommendations

Info

Pointwise Mutual Information EquationCorpus LinguisticsApplication #2:CollocationsMutual Information exampleCorpus LinguisticsApplication #2:CollocationsOur probabilities (p(w 1 w 2 ), p(w 1 ), p(w 2 )) are all basicallycalculated in the same way:(2) p(x) = C(x)N◮ N is the number of words in the corpus◮ The number of bigrams ≈ the number of unigramsCollocationsDefining a collocationKrishnamurthyCalculatingcollocationsPractical workWe want to know if Ayatollah Ruhollah is a collocation in adata set we have:◮ C(Ayatollah) = 42◮ C(Ruhollah) = 20◮ C(AyatollahRuhollah) = 20◮ N = 14, 307, 668CollocationsDefining a collocationKrishnamurthyCalculatingcollocationsPractical work(3) I(w 1 , w 2 ) = log p(w1w2)p(w 1)p(w 2)= logC(w 1 w 2 )NC(w 1 ) C(w 2 )N N= log[N C(w1w2)C(w 1)C(w 2) ] 19 / 28(4) I(Ayatollah, Ruhollah) = log 220N18.3842N × 20N= log 2 N 2042×20 ≈To see how good a collocation this is, we need to compare itto others20 / 28Problems for Mutual InformationCorpus LinguisticsApplication #2:CollocationsMotivating Contingency TablesCorpus LinguisticsApplication #2:CollocationsThe formula we have also has the following equivalencies:(5) I(w 1 , w 2 ) = log p(w1w2)P(w1|w2)p(w 1)p(w 2)= logP(w 1)= log P(w2|w1)P(w 2)CollocationsDefining a collocationKrishnamurthyCalculatingcollocationsWhat we can instead get at is: which bigrams are likely, outof a range of possibilities?CollocationsDefining a collocationKrishnamurthyCalculatingcollocationsMutual information tells us how much more information wehave for a word, knowing the other word◮ But a decrease in uncertainty isn’t quite right ...A few problems:◮ Sparse data: infrequent bigrams for infrequent wordsget high scores◮ Tends to measure independence (value of 0) betterthan dependence◮ Doesn’t account for how often the words do not appeartogether (M&S 1999, table 5.15)Practical workLooking at the Arthur Conan Doyle story A Case of Identity,we find the following possibilities for one particular bigram:◮ sherlock followed by holmes◮ sherlock followed by some word other than holmes◮ some word other than sherlock preceding holmes◮ two words: the first not being sherlock, the second notbeing holmesThese are all the relevant situations for examining thisbigramPractical work21 / 2822 / 28Contingency TablesCorpus LinguisticsApplication #2:CollocationsObserved bigram probabilitiesCorpus LinguisticsApplication #2:CollocationsCollocationsCollocationsWe can count up these different possibilities and put theminto a contingency table (or 2x2 table)B = holmes B holmes TotalA = sherlock 7 0 7A sherlock 39 7059 7098Total 46 7059 7105The Total row and Total column are the marginals◮ The values in this chart are the observed frequencies(f o )Defining a collocationKrishnamurthyCalculatingcollocationsPractical workBecause each cell indicates a bigram, divide each of thecells by the total number of bigrams (7105) to getprobabilities:holmes ¬ holmes Totalsherlock 0.00099 0.0 0.00099¬ sherlock 0.00549 0.99353 0.99901Total 0.00647 0.99353 1.0The marginal probabilities indicate the probabilities for agiven word, e.g., p(sherlock) = 0.00099 andp(holmes) = 0.00647Defining a collocationKrishnamurthyCalculatingcollocationsPractical work23 / 2824 / 28
Expected bigram probabilitiesCorpus LinguisticsApplication #2:CollocationsExpected bigram frequenciesCorpus LinguisticsApplication #2:CollocationsIf we assumed that sherlock and holmes areindependent—i.e., the probability of one is unaffected by theprobability of the other—we would get the following table:holmes ¬ holmes Totalsherlock 0.00647 x 0.00099 0.99353 x 0.00099 0.00099¬ sherlock 0.00647 x 0.99901 0.99353 x 0.99901 0.99901Total 0.00647 0.99353 1.0◮ This is simply p e (w 1 , w 2 ) = p(w 1 )p(w 2 )CollocationsDefining a collocationKrishnamurthyCalculatingcollocationsPractical workMultiplying by 7105 (the total number of bigrams) gives usthe expected number of times we should see each bigram:holmes ¬ holmes Totalsherlock 0.05 6.95 7¬ sherlock 45.5 7052.05 7098Total 46 7059 7105◮ The values in this chart are the expected frequencies(f e )CollocationsDefining a collocationKrishnamurthyCalculatingcollocationsPractical work25 / 2826 / 28Pearson’s chi-square testCorpus LinguisticsApplication #2:CollocationsWorking with collocationsCorpus LinguisticsApplication #2:CollocationsThe chi-square (χ 2 ) test measures how far the observedvalues are from the expected values:(6) χ 2 = ∑ (f o−f e) 2f e(7)χ 2= (7−0.05)20.05+ (0−6.95)26.95+ (39−45.5)245.5+ (7059−7052.05)27052.05CollocationsDefining a collocationKrishnamurthyCalculatingcollocationsPractical workThe question is:◮ What significant collocations are there that start with theword sweet?◮ Specifically, what nouns tend to co-occur after sweet?CollocationsDefining a collocationKrishnamurthyCalculatingcollocationsPractical work= 966.05 + 6.95 + 1.048 + 0.006What do your intuitions say?= 974.05Next time, we will work on how to calculate collocations ...If you look this up in a table, you’ll see that it’s unlikely to bechanceNB: The χ 2 test does not work well for rare events, i.e., f e < 627 / 2828 / 28
Page 1 and 2: Corpus LinguisticsApplication #2:Co
Page 3: Corpus linguisticsKrishnamurthy 200

Collocation Collocations & colligations Defining ... - Indiana University

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?