The dissertation of Andreas Stolcke is approved: University of ...

CHAPTER 4. STOCHASTIC CONTEXT-FREE GRAMMARS 101A.S --> NP VPVP --> VerbI--> VerbT NPNP --> Art Noun--> Art AP NounAP --> Adj--> Adj APArt --> theVerbI --> ate | sleptVerbT --> saw | heardNoun --> cat | dogAdj --> big oldB.S --> NP VPVP --> Verb NPNP --> Art Noun--> Art Noun RCRC --> Rel VPVerb --> saw | heardNoun --> cat | dog | mouseArt --> a | theRel --> thatFigure 4.2: Example grammars from Langley (1994).4.5.3 Sample orderingWe conclude with the two pseudo-natural language examples from Langley (1994), where a hillclimbing learner for non-probabilistic CFGs is studied (see Section 4.4). The point here will be that theordering of samples during incremental processing can have a considerable effect on the outcome, or on thespeed of the learning process.The grammars shown in Figure 4.2 exhibit two types of recursion. Grammar A contains an NP rulefor an arbitrary number of adjectives, such as inthe old big cat heard the big old big dogGrammar B is similar but allows embedded relative clauses instead of adjectives. It yields sentences such asa mouse heard a dog that heard the mouse that heard the catFor the purpose of our experiment uniform probabilitieswere added to the original non-probabilisticCFG productions, and were used to generate 100 random samples each (sample A and B, respectively). Thesamples were presented in one of two conditions: random order, or in order of increasing sentence length(number of words). 15 Both samples were processed by incremental merging and chunking.Sample A ordered by sentence length resulted in a grammar that contained three redundant rulesthat could be removed by reestimation. The final grammar was:S --> NP VI (57)--> NP VT NP (43)NP --> DET N (67)--> DET NP1 (76)NP1 --> ADJ N (76)--> ADJ NP1 (71)DET --> the (143)N --> cat (73)15 The order among samples of the same length remained random.