The dissertation of Andreas Stolcke is approved: University of ...

CHAPTER 4. STOCHASTIC CONTEXT-FREE GRAMMARS 86Sample strings can be enriched to convey some of the phrase structure information. Bracketedsamples are ordinary samples with substrings enclosed by balanced, non-overlapping pairs of parentheses.The bracketing need not be complete. For example, a partially bracketed sample for the example language ofSection 4.3.2 is(a a (a b) b b)It is known that access to completely bracketed samples (equivalent to unlabeled derivation trees)makes learning non-probabilistic CFGs possible and tractable, by applying techniques borrowed from finitestatemodel induction (Sakakibara 1990). Pereira & Schabes (1992) have shown that providing even partialbracketing information can help the induction of properly structured SCFGs using the standard estimationapproach. This raises the question how bracketed samples can be incorporated into the merging algorithmdescribed so far.This is indeed possible by a simple extension of the sample incorporation procedure describedabove. Instead of creating a single top-level production to account for a new sample, the algorithm createsa collection of productions and nonterminals to mirror the bracketing observed. Thus the sample (a a (ab) b b) is added to grammar using the productionsS --> A1 A2 X B2 B3 (1)X --> A3 B1 (1)plus lexical productions for the preterminals created. Each pair or parentheses generates an intermediatenonterminal, such as X above.Merging and chunking are then applied to the resulting grammar as before. If the provided samplebracketing is complete, i.e., contains brackets for all phrase boundaries in the target grammar, then chunkingbecomes unnecessary. Merging alone can in principle produce the target grammar in this case, providedsamples for all productions are given.4.3.4 SCFG priorsIn choosing prior distributions for SCFGs we again extend various approaches previously used forHMMs. As before, a model 4is decomposed into a discrete structure 4 Ã and collection of continuousparameters U Ä . Again, we have a choice of narrow or broad parameter priors, depending on whether theidentity of non-zero rule probabilities is part of 4ùÃ or U Ä (Section 3.3.3). However, notice that broadparameter priors become problematic here since the set of possible rules (and hence parameters) is not apriori limited as for HMMs. As the length of RHSs grows, the number of potential rules over a given set ofnonterminals also grows (exponentially). This makes narrow parameter priors inherently simpler and morenatural for SCFGs.The following combination of priors was used in the experiments reported below. The parameterprior +-,;U Ä . 4TÃE0 is a product of Dirichlet distributions (Section 2.5.5.1), one for each nonterminal. EachDirichlet thus allocates prior probability over all possible expansions of a single nonterminal. The total prior