Perplexity of n-gram and dependency language models

More documents

Recommendations

Info

Language Models – basics P(s) = P(w 1, w 2, ... w m) ≈ Π i=1..m P(w i | w i-2, w i-1) In general: Π i=1..m P(w i | h i ) h i … context (history) <strong>of</strong> word w i 12
Language Models – design decisions 1) How to factorize P(w 1, w 2, ... w m) into Π i=1..m P(w i | h i), i.e. what word-positions will be used as the context h i ? 2) What additional context information will be used (apart from word forms), e.g. stems, lemmata, POS tags, word classes,... ? 3) How to estimate P(w i | h i ) from the training data? Which smoothing technique will be used? (Good-Turing, Jelinek-Mercer, Katz, Kneser-Ney,...) Generalized Parallel Back<strong>of</strong>f etc. 13
Page 1 and 2: Perplexity of n-gram and dependency
Page 3 and 4: P(s) = ? Language Models - basics P
Page 5 and 6: Language Models - basics P(s) = P(w
Page 11: Language Models - basics P(s) = P(w
Page 15 and 16: Language Models - design decisions
Page 17 and 18: Post-ngram LM In general: P(s) = P(
Page 19 and 20: Post-ngram LM In general: P(s) = P(
Page 21 and 22: Dependency LM ● exploit the topol
Page 23 and 24: Dependency LM ● exploit the topol
Page 25 and 26: Dependency LM Motivation for usage
Page 27 and 28: ● Model wp word form of parent P(
Page 29 and 30: ● Model E,wp edge direction, word
Page 31 and 32: ● Model N,wp the word is N th chi
Page 33 and 34: Dependency LM Examples of additiona
Page 35 and 36: Dependency LM Examples of additiona
Page 37 and 38: Evaluation ● Train and test data
Page 39 and 40: Findings confirmed for all seven la
Page 41: Thank you 41

Perplexity of n-gram and dependency language models

Create successful ePaper yourself

Delete template?

Save as template?