The dissertation of Andreas Stolcke is approved: University of ...

ICHAPTER 4. STOCHASTIC CONTEXT-FREE GRAMMARS 95the number caused the resulting grammars to be less general in several cases, and giving significantly lesssamples (10) would produce over-general results. We already discussed why an evaluation function should,in fact, depend on the actual number of samples, not just their relative frequencies, and one can conclude thatCook’s evaluation function is tuned to be roughly equivalent to the Bayesian posterior for a sample countaround 50.Palindromes The simple palindrome (!(eXŽ(languagebenchmark language in several investigations into CFG learning algorithms. Cook et al. (1976) discuss itE# has proven to be a relatively difficultbriefly as a language beyond the reach of their search operators, noting that it would require a chunkingoperator that does not blindly replace all occurrences of a chunk. To learn this grammar, one has to startchunking and , sequences that typically also occur in non-center position, thereby misleading a strictlygreedy learning strategy.Pereira & Schabes (1992) use the same language to show that standard SCFG estimation does verypoorly in finding a grammar of the right form when started with random initial parameters. The results ofestimation can be improved considerably by using bracketed samples instead (although the resulting grammarstill gives incorrect parse structure to some strings).Indeed our algorithm fails to find a good grammar for this language using only best-first search, dueto the limitations of the chunking operation cited by Cook. 12 However, the result improved when applyingthe more powerful beam search (beam width 3).S --> A A--> B B--> A S A--> B S B--> S S ***A --> aB --> bThis is almost a perfect grammar for the language, except for the production marked by ***. This productionis redundant (not strictly required for the derivation of any of the samples) and can be eliminated by a simpleViterbi reestimation step following the search process. (The Viterbi step is meant to correct counts that havebecome inaccurate due to the optimistic update procedure during merging.)While this result in itself is not very significant it does show that the simple merging operatorsbecome considerably more powerful when combined with more sophisticated search techniques, as expected.merging only.Incidentally, the palindrome language becomes very easy to learn from bracketed samples, using12 For concreteness, we used the same training setup as for the easier palindrome language with center symbol (fifth in Table 4.1). Thesamples and their number where identical except for removal of the center marker « .