The dissertation of Andreas Stolcke is approved: University of ...

75Chapter 4Stochastic Context-free Grammars4.1 Introduction and OverviewIn this chapter we will look at model merging as applied to the probabilistic version of context-freegrammars. The stochastic context-free grammar (SCFG) formalism is a generalization of the HMM, just asnon-probabilistic CFGs can be thought of as an extension of finite state grammars.Unlike their their non-probabilisticcounterpart, SCFGs are not a ‘mainstream’ approach to languagemodeling yet. 1 In most of today’s probabilisticlanguage models finite-state or even simple ¢ -gram approachesdominate. One reason for this is that although most standard algorithms for probabilistic finite-state models(i.e., HMMs) have generalized versions for SCFGs, they become computationally more demanding, and oftenintractable in practice (see Section 4.2.2).A more important problem is that SCFGs may actually be worse at modeling one aspect of languagein which simple finite-state models do a surprisingly good job: capturing the short-distance, lexical (asopposed to phrase-structural) contingencies between words. This is a direct consequence of the conditionalindependence assumptions embodied in SCFGs, and has prompted the investigation of ‘mildly contextsensitive’grammars and their probabilistic versions (Resnik 1992; Schabes 1992). These, however, come atan even greater computational price.domain.Recent work has shown that probabilistic CFGs can be useful if applied carefully and in the rightLari & Young (1991) discuss various applications of estimated SCFGs for phonetic modeling.Jurafsky et al. (1994b) show that a SCFG built from hand-crafted rules with probabilities estimated from acorpus can improve speech recognition performance over standard ¢ -gram language models, either by directlycoupling the SCFG to the speech decoder, or by using the SCFG effectively as a smoothing device to improvethe estimates of ¢ -gram probabilities from sparse data. The algorithms that form the basis of these last twoapproaches are described in the second part of this thesis, in Chapter 6 and Chapter 7, respectively.1 While bare CFGs aren’t widely used in computational linguistics either, they form the basis or ‘backbone’ of most of today’sfeature and unification-based grammar formalisms, such as LFG (Kaplan & Bresnan 1982), GPSG (Gazdar et al. 1985), and constructiongrammar (Fillmore 1988).