Statistical Language Models based on Neural Networks - Faculty of ...

More documents

Recommendations

Info

5-gram: HE SAID SUEZ AIMED TO BRING BETTER MANAGEMENT OF THE COMPANY TO INCREASE PRODUCTIVITY AND PROFITABILITY RNN: HE SAID SUEZ AIMED TO BRING BETTER MANAGEMENT TO THE COMPANY TO INCREASE PRODUCTIVITY AND PROFITABILITY 5-gram: JOSEPH A. M. G. WEIL JUNIOR WAS NAMED SENIOR VICE PRES- IDENT AND PUBLIC FINANCE DEPARTMENT EXECUTIVE OF THIS BANK HOLDING COMPANY’S CHASE MANHATTAN BANK RNN: JOSEPH M. JAKE LEO JUNIOR WAS NAMED SENIOR VICE PRESIDENT AND PUBLIC FINANCE DEPARTMENT EXECUTIVE OF THIS BANK HOLD- ING COMPANY’S CHASE MANHATTAN BANK 5-gram: IN THE NEW LEE CREATED POSITION HE HEADS THE NEW PUBLIC FINANCE DEPARTMENT RNN: IN THE NEW LEE KOREAN POSITION HE HEADS THE NEW PUBLIC FINANCE DEPARTMENT 5-gram: MR. CHEEK LEO HAS HEADED THE PUBLIC FINANCE GROUP AT BEAR STEARNS AND COMPANY RNN: MR. JAKE LEO HAS HEADED THE PUBLIC FINANCE GROUP AT BEAR STEARNS AND COMPANY 5-gram: PURCHASERS ALSO NAMED A ONE HUNDRED EIGHTY NINE COM- MODITIES THAT ROSE IN PRICE LAST MONTH WHILE ONLY THREE DROPPED IN PRICE RNN: PURCHASERS ALSO NAMED ONE HUNDRED EIGHTY NINE COMMODI- TIES THAT ROSE IN PRICE LAST MONTH WHILE ONLY THREE DROPPED IN PRICE 5-gram: ONLY THREE OF THE NINE BANKS SAW FOREIGN EXCHANGE PROF- ITS DECLINED IN THE LATEST QUARTER RNN: ONLY THREE OF THE NINE BANKS SAW FOREIGN EXCHANGE PROF- ITS DECLINE IN THE LATEST QUARTER 5-gram: THE STEEPEST FALL WAS THE BANKAMERICA COURTS BANK OF AMERICA A THIRTY PERCENT DECLINE TO TWENTY EIGHT MILLION DOLLARS FROM FORTY MILLION DOLLARS RNN: THE STEEPEST FALL WAS A BANKAMERICA COURT’S BANK OF AMER- ICA A THIRTY PERCENT DECLINE TO TWENTY EIGHT MILLION DOLLARS FROM FORTY MILLION DOLLARS 5-gram: A SPOKESWOMAN BLAMED THE DECLINE ON MARKET VOLATIL- ITY AND SAYS THIS SWING IS WITHIN A REASONABLE RANGE FOR US RNN: A SPOKESWOMAN BLAMES THE DECLINE ON MARKET VOLATILITY 128
AND SAYS THIS SWING IS WITHIN A REASONABLE RANGE FOR US 5-gram: LAW ENFORCEMENT OFFICIALS SAID SIMPLY MEASURE OF THEIR SUCCESS BY THE PRICE OF DRUGS ON THE STREET RNN: LAW ENFORCEMENT OFFICIALS SAID SIMPLY MEASURE THEIR SUC- CESS BY THE PRICE OF DRUGS ON THE STREET 5-gram: IF THE DRY UP THE SUPPLY THE PRICES RISE RNN: IF THEY DRY UP THE SUPPLY THE PRICES RISE 5-gram: CAROLYN PRICES HAVE SHOWN SOME EFFECT FROM THE PIZZA SUCCESS AND OTHER DEALER BLASTS RNN: CAROLYN PRICES HAVE SHOWN SOME EFFECT ON THE PIZZA SUC- CESS AND OTHER DEALER BLASTS 129
Page 1 and 2:
VYSOKÉ UČENÍ TECHNICKÉ V BRNĚ
Page 3 and 4:
Abstrakt Statistické jazykové mod
Page 5 and 6:
Contents 1 Introduction 4 1.1 Motiv
Page 7 and 8:
6.2.3 Reduction of Vocabulary Size
Page 9 and 10:
Maybe the most popular vision of fu
Page 11 and 12:
Chapter 6 presents further extensio
Page 13 and 14:
Chapter 2 Overview of Stati
Page 15 and 16:
2.1 Evaluation 2.1.1 Perplexity Eva
Page 17 and 18:
ALP can be used to obtain prior pro
Page 19 and 20:
• Good theoretical motivation •
Page 21 and 22:
abilities of n-grams are stored in
Page 23 and 24:
main (static) n-gram model. As the
Page 25 and 26:
There are many popular examples sho
Page 27 and 28:
y Chen et al., who proposed a so-ca
Page 29 and 30:
confusion among researchers, and ma
Page 31 and 32:
language model took almost a week u
Page 33 and 34:
w(t) s(t-1) s(t) U V W y(t) Figure
Page 35 and 36:
ate is halved at start of every new
Page 37 and 38:
or using matrix-vector notation as
Page 39 and 40:
information for more than 5 time st
Page 41 and 42:
A simple solution to the exploding
Page 43 and 44:
output layer changes to computation
Page 45 and 46:
While RNN models can overcome this
Page 47 and 48:
complex or random architectures (su
Page 49 and 50:
While for any of the previous point
Page 51 and 52:
where λ is the interpolation weigh
Page 53 and 54:
model with default SRILM cutoffs pr
Page 55 and 56:
experiments, we have used the one i
Page 57 and 58:
Perplexity (Penn corpus) 145 140 13
Page 59 and 60:
with syntactical NNLMs would be pre
Page 61 and 62:
Table 4.3: Combination of individua
Page 63 and 64:
Table 4.6: Results on Penn Treebank
Page 65 and 66:
4.6 Conclusion of the Model Combina
Page 67 and 68:
were: 400 classes, hidden layer siz
Page 69 and 70:
Entropy per word on the WSJ test da
Page 71 and 72:
Table 5.3: Results on the WSJ setup
Page 73 and 74:
Table 5.5: Results for models <stro
Page 75 and 76:
trained together with a maximum ent
Page 77 and 78:
wt-3 wt-2 wt-1 D D D P(wt|context)
Page 79 and 80:
n-gram probabilities. However, it w
Page 81 and 82: Table 6.1: Training corpora for NIS
Page 83 and 84: Perplexity 360 340 320 300 280 260
Page 85 and 86: Entropy per word 9 8.5 8 7.5 7 6.5
Page 87 and 88: 1 a a a 1 2 3 P(w(t)|*) ONE TWO THR
Page 89 and 90: Table 6.4: Perplexity on the evalua
Page 91 and 92: Entropy reduction per word over KN4
Page 93 and 94: Table 6.6: Perplexity with the new
Page 95 and 96: Entropy reduction over KN5 -0.04 -0
Page 97 and 98: as a baseline, and 12.3% after resc
Page 99 and 100: Table 7.1: BLEU on IWSLT 2005 Machi
Page 101 and 102: Table 7.3: Size of compressed text
Page 103 and 104: Table 7.4: Accuracy of different la
Page 105 and 106: Table 7.6: Entropy on PTB with n-gr
Page 107 and 108: 8.1 Machine Learning One possible d
Page 109 and 110: that almost every non-trivial compu
Page 111 and 112: supervision such as one digit at a
Page 113 and 114: Chapter 9 Conclusion and Future Wor
Page 115 and 116: from the expensive part of the mode
Page 117 and 118: Bibliography [1] A. Alexandrescu, K
Page 119 and 120: [23] D. Filimonov, M. Harper. A joi
Page 121 and 122: [50] T. Mikolov, S. Kombrink, L. Bu
Page 123 and 124: [77] W. Wang, M. Harper. The SuperA
Page 125 and 126: Test Phase After the model is train
Page 127 and 128: • compute sentence-level scores g
Page 129 and 130: Appendix B: Data generated from mod
Page 131: Appendix C: Example of decoded utte
show all

Statistical Language Models based on Neural Networks - Faculty of ...

Create successful ePaper yourself

Delete template?

Save as template?