The dissertation of Andreas Stolcke is approved: University of ...

A prerequisite for this approach is to precompute, for all nonterminals ) , the probability that )”””Ó”¸=1=2òò””CHAPTER 6. EFFICIENT PARSING WITH STOCHASTIC CONTEXT-FREE GRAMMARS 141section summarizes the necessary modifications to process null productions correctly, using the previousdescription as a baseline. Our treatment of null productions follows the (non-probabilistic) formulation ofGraham et al. (1980), rather than the original one in Earley (1970).6.4.7.1 ComputingA-expansion probabilitiesThe main problem with null productions is that they allow multiple prediction-completion cyclesin between the scanning steps (since null productions do not have to be matched against one or more inputsymbols). Our strategy will be to collapse all predictions and completions due to chains of null productionsinto the regular prediction and completion steps, not unlike the way recursive predictions/completions werehandled in Section 6.4.5.expands to the empty string. Note that this is another recursive problem, ) sinceproduction, but expand to nonterminal= some that does.Computation ) of for ) allFor +-, each , let be an abbreviation for +-, C«A0)Ó )itself may not have a nullcan be cast as a system of non-linear equations, as follows.. By way of example, if ) CEA0 has productions) ¸ Atò1ót 2ó.¸=3=4=5t 3óThe semantics of context-free rules imply that ) can only expand toAif all of the RHSs in one of ) ’sproductions expands toA.Translating to probabilities, we obtain the equation6/t 1 Ngt 2Ó‘1Ó‘2Ngt 3Ó‘3Ó‘4Ó‘5N|£££In other words, each production contributes a term in which the rule probability is multiplied by the productof the Ó variables corresponding to the RHS nonterminals, unless the RHS contains a terminal (in which casethe production contributes nothing Ó to because it cannot possibly lead toA).The resulting non-linear system can be solved by iterative approximation. Each Ó variableinitialized to +-, ) ¸A0 , and then repeatedly updated by substituting in the equation right-hand sides, untilisincreasing and bounded above by the true values +9, )the desired level of accuracy is attained. Convergence is guaranteed since Ó the values are monotonically1. For grammars without cyclic dependencies CEA0ŒVamongA-producing nonterminals this procedure degenerates to simple backward substitution. Obviously thesystem has to be solved only once for each grammar.The Ó probability can be seen as the precomputed inner probability of an expansion ) of to theempty string, i.e., it sums the probabilities of all Earley paths that ) deriveAfrom . This is the justificationfor the way these probabilities can be used in modified prediction and completion steps, described next.