The dissertation of Andreas Stolcke is approved: University of ...

H : d§=:6) ¸Æ¸¸ 99W9 Consequently, the example trace shows that the factortL++?16ò u ó°£?,,ò:6) ¸ .„>£16?1󰣘ŠÙ ó ÙÌCHAPTER 6. EFFICIENT PARSING WITH STOCHASTIC CONTEXT-FREE GRAMMARS 139The existence of ©¦ is shown in Section 6.4.8.The modified completion loop in the probabilistic Earley parser can now use the © matrix tocollapse all unit completions into a single step. Note that we still have to do iterative completion on non-unitproductions.Completion (probabilistic, transitive)ò…Æ@$£Ù…Ù )ŠÙ…Ù óò…Ƹò Æ.—£>Θ)ŠóŽ 6QC Hfor all=i?> such that ©,> C©=ã0 isnon-zero, and=¸@is not a unit production (.@%.1). ThenÙ += ÆŠÙ…Ù © C©=ë0 ,>6.4.6 An exampleŠÙ += Š}ŠÙ…Ù © Cª=90 ,>Consider the grammar9tróu ówhere u 6 1 L"t . This highly ambiguous grammar generates strings of any number of ’s, using all possiblebinary parse trees over the given number of terminals. The states involved in parsing the string are listedin Table 6.2, along with their forward and inner probabilities. The example illustrates how the parser dealswith left-recursion and the merging of alternative sub-parses during completion.Since the grammar has only a single nonterminal, the left-corner matrix + has rank 1:ò 6Its transitive closure is©"6µ0?tóò,;²in a number of prediction steps.The sample string can be parsed as either , 070 or ,_, 070 , each parse having a probability of1 being introduced into the forward probability termstòt 3u 2 . The total string probability is thus 2t 3u 2 , the computed Æ andŠ values for the final state. The Ævalues for the scanned states in sets 1, 2 and 3 are the prefix probabilities for , , and , respectively: 1, +-,9 CTg0%6 u, +-,9 CÀg0%6 CÀh0%6+-,91 NËtE0 u 2 .6.4.7 Null productionsNull productions ) ¸Aintroduce some complications into the relatively straightforward parseroperation described so far, some of which are due specifically to the probabilistic aspects of parsing. This