The dissertation of Andreas Stolcke is approved: University of ...

CHAPTER 6. EFFICIENT PARSING WITH STOCHASTIC CONTEXT-FREE GRAMMARS 164exact SCFG prefix and next-word probabilities to a tightly-coupled speech decoder (Jurafsky et al. 1994b).An essential idea in the probabilistic formulation of Earley’s algorithm is the collapsing of recursivepredictions and unit completion chains, replacing them by lookups in precomputed matrices. This idea arisesin our formulation out of the need to compute probability sums given as infinite series. Graham et al. (1980)use a non-probabilistic version of the same technique to create a highly optimized Earley-like parser forgeneral CFGs that implements prediction and completion by operations on Boolean matrices. 25The matrix inversion method for dealing with left-recursive prediction is borrowed from the LRIalgorithm of Jelinek & Lafferty (1991) for computing prefix probabilities for SCFGs in CNF. 26 We then usethat idea a second time to deal with the similar recursion arising from unit productions in the completion step.We suspect, but have not proved, that the Earley computation of forward probabilities when applied to a CNFgrammar performs a computation that is in some sense isomorphic to that of the LRI algorithm. In any case,we believe that the parser-oriented view afforded by the Earley framework makes for a more intuitive solutionto the prefix probability problem, with the added advantage that it is not restricted to CNF grammars.Kupiec (1992a) has proposed a version of the Inside/Outside algorithm that allows it to operateon non-CNF grammars. Interestingly, Kupiec’s algorithm is also based on a generalization of finite-statemodels, namely, Recursive Transition Networks (RTNs). Probabilistic RTNs are essentially HMMs thatallow nonterminals as output symbols. Also, the dotted productions appearing in Earley states are exactlyequivalent to the states in an RTN derived from a CFG.6.7.5 A simple typology of SCFG algorithmsThe various known algorithms for probabilistic CFGs share many similarities, and vary alongsimilar dimensions. One such dimension is whether the quantities entered into the parser chart are defined ina bottom-up (CYK) fashion, or whether left-to-right constraints are an inherent part of their definition. 27Another point of variation is the ‘sparseness’ trade-off. If we are given a set of nonterminals andwanted to list all possible CFG rules involvingthose nonterminals, the list would be infinite due to the arbitrarylength of the right-hand sides of productions. This is a problem, for example, when training a CFG startingwith complete ignorance about the structure of the rules.A workaround is to restrict the rule format somehow, usually to CNF, and then list all possibleproductions. Algorithms that assume CNF are usually formulated in terms of such a fully parameterizedgrammar where all triples )g)=i?> form a possible rule ) ¸ =>withcases they may be specialized to handle sparse grammars efficiently.non-zero probability, although in manyAt the other extreme we have algorithms with accept unrestricted CFG productionsand are thereforemeant for sparse grammars, where almost all (in the set theoretic sense) possible productions have probability25 This connection to the GHR algorithm was pointed out by Fernando Pereira. Exploration of this link then lead to the extension ofour algorithm to handle¹-productions, as described in Section 6.4.7.26 Their method uses the transitive (but not reflexive) closure over the left-corner relation ¦Vº, for which they chose the symbol»º.We chose the symbol”ºin this chapter to point to this difference.27 Of course a CYK-style parser can operate left-to-right, right-to-left, or otherwise by reordering the computation of chart entries.