The dissertation of Andreas Stolcke is approved: University of ...

) The probabilistic unit-production relation + 6c) The relation ) ,©,+,{§0 is the matrix of probabilities +9, ) ¸defined as the reflexive, transitive closure of ) ¸=, i.e., ) ,6NöNN yž1,and>L+?+-,>1 ¸+-,>1 ¸ >20+-,>2 ¸iff) 6L=CHAPTER 6. EFFICIENT PARSING WITH STOCHASTIC CONTEXT-FREE GRAMMARS 138Lemma 6.4 Letbe a completion i.e.,"16§"] cycle, ) , 6Ð) ] 1.1 6m.2 ,.16§.]65£££6m.]1 .1) 2£s6QC 62) 2 ¸ .2) 6¦) 3£s6QC £££6QC 61) ¸ , 6Ð) ] 2 )all productions involved are ) unit productions ) 2£££_) ] ¸ ) ]1 ¸1. Then it must be the case that6mA,##i.e., 1.Proof. For all completion chains it is true that the start indices of the states are monotonically6same input index in all states, all ) 1_) 2£££7_) ] nonterminals have been expanded into the same substring(a state can only complete an expansion that started at the same or a previousincreasing,"1‹¨"2‹Ð£££position). From"1 6L"]follows that"1 6L"2 . Because the current position (dot) also refers to the£££6#"]of the input between"1 and the current position. By assumption the grammar contains no nonterminals thatgenerateA, 13 therefore we must have.1 6m.2 6LA, q.e.d.6x£££6m.]We now formally define the relation between nonterminals mediated by unit productions, analogousto the left-corner relation.].] ) ] ¸ 1£ #Definition 6.6 The following definitions are relative to a given SCFG{.a) Two nonterminals ) and= are said to be in a unit-production relation ) ¸ =production for )that has= as its RHS.there exists a=ë0 .or there is a nonterminal> such that ) ¸ >C©=.sums © . Each © C©=ë0 ) is C©=ë0 )defined as a seriesC~= isC~= iffd) The probabilisticreflexive, transitive unit-productionrelation ©ëÜ6 ©¦,{§0 is the matrix of probability+-, C©=906 ))Ð6#=ë0) ¸=ë0+-,ž1¢ž2+-,) ¸ >10 y+-,) ¸ >10=ë0=90N/£££)g)=ë0MN yž>!0_© C©=-0£,>As before, a matrix inversion can compute the relation ©¤ in closed form:) ¸ +-,,;²1£13 ©¦Ü6 '0Even with null productions, these would not be used for Earley transitions, see Section 6.4.7.