The dissertation of Andreas Stolcke is approved: University of ...

)ÅÆÅÅÅÆÅÅÅÆÅÅÅÆÅÅÅÆÅÅÅÆÅÅÅ= >the string Æ as a suffix, while )occurrences of ({65( 1 £££;(,,===Æ)= >means that ) generates Æ as a prefix. +-, ) ,Æ)= >in strings generated by an arbitrary nonterminal ) . The special case ===,0CHAPTER 7. -GRAMS FROM STOCHASTIC CONTEXT-FREE GRAMMARS 172((a) (b) (c)( (ÅÆÅÅÅÆÅÅÅÆÅÅÅÆÅÅÅÆÅÅÅÆÅÅÅ ÅÆÅÅÅÆÅÅÅÆÅÅÅÆÅÅÅÆÅÅÅÆÅÅÅFigure 7.1: Three ways of generating a substring ( from a nonterminal ) .Notation Extending the notation used in ) previousÆchapters,X Cdenotes that non-terminal) generatesare the probabilities associated with these events.C/and ) 0ÆX +9, CCT 7.3.3 Computing expectationsOur goal now is to compute the substring expectations for a given grammar. Formalisms such asSCFGs which have a recursive rule structure suggest a divide-and-conquer algorithm that follows the recursivestructure of the grammar.We generalize the problem by considering (ë. )g0 , the expected number of (possibly overlapping)is the solution sought, where9 is the start symbol of the grammar.Now consider all possible nonterminal) ways that can (/6|( generate £££7(string 11 £££7( £££ , and the associated probabilities. For each production of ) CF£££;( (ë.9%0as a substring,denoted ) by we have todistinguish two main cases, assuming the grammar is in CNF. If the string in question is of length (6{( 1, 1,and if ) happens to have a production ) ¸ ( 1, then that production adds exactly +-, ) ¸ ( 10 to the expectation( then might also be generated by recursiveexpansion of the right-hand side. Here, for each production, there are three subcases.If ) has non-terminal productions, say, ) ¸ =>,(-. )g0 .(a) First,= can by itself generate the complete ( (see Figure 7.1(a)).(b) Likewise,> itself can generate ( (Figure 7.1(b)).Finally,= (c) could ( generate £££_( d 1 as a suffix (=), thereby resulting in a single occurrence of ( (Figure 7.1(c)).1 £££;( d ) and>, ( d #1 £££;(as a prefixexpectation (Each of these cases will have an expectation for generating ( 1 £££;(X Cas a substring, and the total(ë. )g0 will be the sum of these partial expectations. The total expectations for the first two C¬(%d #1 £££7((>