The dissertation of Andreas Stolcke is approved: University of ...

,?Ó,tÌ?L A0,I 1N I N A A 2 A 3 N £££ N 6CHAPTER 7. -GRAMS FROM STOCHASTIC CONTEXT-FREE GRAMMARS 176This leads toÍ6 6tL 2t-L £ t/Ï t¬6 0£ 0£1 2 1 (7.10)Now, for 5 this becomes infinity, and for probabilities 5, the solution is negative! This is arather striking manifestation of the failure of this grammar, for tKV 0£ 5, to be consistent in the sense of Booth& Thompson (1973) (see Section 6.4.8). An inconsistent grammar is one in which the stochastic derivationprocess has non-zero probability of not terminating. The expected length of the generated strings shouldtherefore be infinite in this case.Booth and Thompson derive a criterion for checking the consistency of a SCFG: Find the firstmomentmatrix 6 E is the expected number of occurrences of nonterminal= in a one-step.”‘, where Ó”‘0expansion of ) nonterminal , and make sure its powers E6converge to as" 0¸Í°consistent, otherwise it is not. 5If so, the grammar isFor the grammar in (7.9), E is the 1 ¾ 1 matrix , 2 u 0 . Thus we can confirm our earlier observationby noting that , 2 u 0|6converges to 0 iff u Ï 0£ 5, or t0£ 5.Notice that E is identical to the matrix A that occurs in the linear equations (7.6) for the ¢ -gramcomputation. The actual coefficient matrix is I L A, and its inverse, if it exists, can be written as the geometricsumThis series converges precisely if A6converges to 0. We have thus shown that the existence of a solutionfor the¢ -gram problem is equivalent to the consistency of the grammar in question. Furthermore, the solution vector6 c L A0 I 1 b will always consist of non-negative numbers: it is the sum and product of the non-negativevalues given by equations (7.7) and (7.8).The matrix L I A and its inverse turn out to have a special role for SCFG: it is, in a sense, a‘universal problem solver’ for a whole series of global quantities associated with probabilistic grammars. Abrief overview of these is given in the appendix to this chapter.7.6 ExperimentsThe algorithm described here has been implemented, and is being used to generate bigrams fora speech recognizer that is part of the BeRP spoken-language system (Jurafsky et al. 1994a). The speechdecoder and language model components of the BeRP system were used in an experiment to assess the benefitof using bigram probabilities obtained through SCFGs versus estimating them directly from the availabletraining corpus. The system’s domain are inquiries about restaurants in the city of Berkeley. Table 7.1 givesstatistics for the training and test corpora used, as well as the language models involved in the experiment.Our experiments made use of a context-free grammar hand-written for the BeRP domain. Computing thebigram probabilities from this SCFG of 133 nonterminals involves solving 657 linear systems for unigram5 An alternative version of this criterion is to check the magnitude of the largest of E’s eigenvalues (its spectral radius). If that valueisÎ 1, the grammar is inconsistent; ifÏ 1, it is consistent.