Perplexity of n-gram and dependency language models

More documents

Recommendations

Info

Language Models – basics P(s) = P(w 1, w 2, ... w m) = P(w 1) P(w 2|w 1) … P(w m|w 1,...,w m-1) P( The dog barked again ) = P(wi = The | i=1, wi-1 = NONE ) · P(wi = | i=2, wi-2 = NONE , wi-1 = The ) · dog Artificial start-<strong>of</strong>-sentence token P(wi = barked | i=3, wi-3 = NONE , wi-2 = The , wi-1 = dog ) · P(wi = again | i=4, wi-4 = NONE , wi-3 = The , wi-2 = dog , wi-1 = barked ) 8
Language Models – basics P(s) = P(w 1, w 2, ... w m) = P(w 1) P(w 2|w 1) … P(w m|w 1,...,w m-1) P( The dog barked again ) ≈ P(wi = The | wi-1 = NONE ) · P(wi = | wi-2 = NONE , wi-1 = The ) · dog Position back<strong>of</strong>f P(wi = barked | wi-3 = NONE , wi-2 = The , wi-1 = dog ) · P(wi = again | wi-4 = NONE , wi-3 = The , wi-2 = dog , wi-1 = barked ) 9
Page 1 and 2: Perplexity of n-gram and dependency
Page 3 and 4: P(s) = ? Language Models - basics P
Page 5 and 6: Language Models - basics P(s) = P(w
Page 7: Language Models - basics P(s) = P(w
Page 11 and 12: Language Models - basics P(s) = P(w
Page 13 and 14: Language Models - design decisions
Page 15 and 16: Language Models - design decisions
Page 17 and 18: Post-ngram LM In general: P(s) = P(
Page 19 and 20: Post-ngram LM In general: P(s) = P(
Page 21 and 22: Dependency LM ● exploit the topol
Page 23 and 24: Dependency LM ● exploit the topol
Page 25 and 26: Dependency LM Motivation for usage
Page 27 and 28: ● Model wp word form of parent P(
Page 29 and 30: ● Model E,wp edge direction, word
Page 31 and 32: ● Model N,wp the word is N th chi
Page 33 and 34: Dependency LM Examples of additiona
Page 35 and 36: Dependency LM Examples of additiona
Page 37 and 38: Evaluation ● Train and test data
Page 39 and 40: Findings confirmed for all seven la
Page 41: Thank you 41

Perplexity of n-gram and dependency language models

Create successful ePaper yourself

Delete template?

Save as template?