The dissertation of Andreas Stolcke is approved: University of ...

-grams CCHAPTER 7. -GRAMS FROM STOCHASTIC CONTEXT-FREE GRAMMARS 174The computation of prefix probabilities for SCFGs is generally useful for applications and has beensolved with the LRI algorithm (Jelinek & Lafferty 1991). In Chapter 6 we have seen how this computation canbe carried out efficiently for sparsely parameterized SCFGs using a probabilistic version of Earley’s parser.Computing suffix probabilities is obviously a symmetrical task; for example, one could create a ‘mirrored’SCFG (reversing the order of right-hand side symbols in all productions) and then run any prefix probabilitycomputation on that mirror grammar.Note that in the case of bigrams, only a particularly simple form of prefix/suffix probabilities arerequired, namely, the ‘left-corner’ ) ( 10 and ‘right-corner’ ( 20 probabilities, and , whichcan each be obtained from a single matrix inversion (Jelinek & Lafferty 1991), corresponding to the left-cornerX C +-,= +-,matrix © used in the probabilistic Earley parser (as well as the corresponding right-corner matrix).Finally, it is interesting to compare the relative ease with which one can solve the substringexpectation problem to the seemingly similar problem of finding substring probabilities: the probability that) generates (one or more instances of) ( . The latter problem is studied by Corazza et al. (1991), and shownto lead to a non-linear system of equations. The crucial difference here is that expectations are additive withrespect to the cases in Figure 7.1, whereas the corresponding probabilities are not, since the three cases canoccur in the same string.7.3.5 Ëcontaining string boundariesA complete ¢ -gram grammar includes strings delimited by a special marker denoting beginningand end of a string. In Section 2.2.2 we had introduced the symbol ‘$’ for this purpose.To generate expectations for ¢ÜL 1-grams adjoining the string boundaries, the original SCFGgrammar is augmented by a new top-level production9Ù¸$9$1£ 0óòis the old start symbol, and9%Ù becomes the start symbol of the augmented grammar. The algorithmis then simply applied to the augmented grammar to give the desired ¢ -gram probabilities including the ‘$’where9marker.7.4 Efficiency and Complexity IssuesSummarizing from the previous section, we can compute any ¢ -gram probability by solving twolinear systems of equations of the form (7.3), one with ( being the ¢ -gram itself and one for the , ¢¦L 10 -gramprefix ( 1 £££;(=$?1. The latter computation can be shared among all ¢ -grams with the same prefix, so thatessentially one system needs to be solved for each ¢ -gram we are interested in. The good news here is thatthe work required is linear in the number ¢ of -grams, and correspondingly limited if one needs probabilitiesfor only a subset of the ¢ possible -grams. For example, one could compute these probabilities on demandand cache the results.