Perplexity of n-gram and dependency language models

More documents

Recommendations

Info

● Language Models (LM) – basics – design decisions ● Post-ngram LM ● Dependency LM ● Evaluation Outline ● Conclusion & future plans 16
Post-ngram LM In general: P(s) = P(w 1 , w 2 , ... w m ) ≈ Π i=1..m P(w i | h i ) h i … context (history) of word w i left-to-right factorization order Bigram LM: h i = w i-1 (one previous word) Trigram LM: h i = w i-2 , w i-1 (two previous words) 17
Page 1 and 2: Perplexity of n-gram and dependency
Page 3 and 4: P(s) = ? Language Models - basics P
Page 5 and 6: Language Models - basics P(s) = P(w
Page 13 and 14: Language Models - design decisions
Page 15: Language Models - design decisions
Page 19 and 20: Post-ngram LM In general: P(s) = P(
Page 21 and 22: Dependency LM ● exploit the topol
Page 23 and 24: Dependency LM ● exploit the topol
Page 25 and 26: Dependency LM Motivation for usage
Page 27 and 28: ● Model wp word form of parent P(
Page 29 and 30: ● Model E,wp edge direction, word
Page 31 and 32: ● Model N,wp the word is N th chi
Page 33 and 34: Dependency LM Examples of additiona
Page 35 and 36: Dependency LM Examples of additiona
Page 37 and 38: Evaluation ● Train and test data
Page 39 and 40: Findings confirmed for all seven la
Page 41: Thank you 41

Perplexity of n-gram and dependency language models

Create successful ePaper yourself

Delete template?

Save as template?