SRI-LM toolkit

18.05.2015 • Views

Share Embed Flag

SRI-LM toolkit

SRI-LM toolkit

DOWNLOAD ePAPER

berlin.csie.ntnu.edu.tw

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

START NOW

Compute perplexity<br />

<strong>LM</strong> 檔<br />

要算 perplexity 的 test data<br />

輸出範<br />

例 :<br />

−<br />

logprob<br />

句數 + 字數<br />

10<br />

字數<br />

10 −logprob<br />

10

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

First-Order Logic and Inference - Berlin Chen

Conditional Probability, Total Probability Theorem and ... - Berlin Chen

Linear Prediction Analysis of Speech Sounds - Berlin Chen

Solving Problems by Searching - Berlin Chen

Introduction to SRILM Toolkit

Retrieval Models (III) - Latent Semantic Analysis (LSA) - Berlin Chen

Large Vocabulary Continuous Speech Recognition - Berlin Chen

Set-Based Model: A New Approach for Information Retrieval

Chapter 2: Discrete Random Variables: Basics - Berlin Chen

Generalized Vector Space Model In Information Retrieval

Supervised Learning - PAC, VC-Dimension etc. - Berlin Chen

Agent - Berlin Chen

Numerical Methods - Berlin Chen

Further Topics on Random Variables: Others - Berlin Chen

Linear Algebra Homework 1

Compute perplexity <strong>LM</strong> 檔要算 perplexity 的 test data 輸出範例 : − logprob 句數 + 字數 10 字數 10 −logprob 10

Good-Turing Discounting Parameters maxcount mincount 自行指定 mincount 跟 maxcount : 11

Page 1 and 2: SRI-LM toolkit 2003/12/16 Louis Tsa
Page 3 and 4: Create language model ~ W = arg max
Page 5 and 6: Vocabulary (Lexicon2003-72k.txt)
Page 7 and 8: 以 \data\ 開頭 LM Format (ssp.n
Page 9: Good-Turing Discounting and 預設
Page 13 and 14: • Take the conditional probabilit
Page 15 and 16: Witten-Bell Discounting • Key Con
Page 17 and 18: Witten-Bell Discounting p * i ci =
Page 19 and 20: • For bigram ∑ p Witten-Bell Di
Page 21 and 22: P abs Smoothing absolute discountin