The dissertation of Andreas Stolcke is approved: University of ...

CHAPTER 5. PROBABILISTIC ATTRIBUTE GRAMMARS 114S --> A CIRCLE IS BELOW A SQUARES.tr = CIRCLE.f1S.lm = SQUARE.f2S.rel = ‘below’CIRCLE --> circleCIRCLE.f1 = ‘circle’SQUARE --> squareSQUARE.f2 = ‘square’The nonterminal merging operation merge(CIRCLE, SQUARE) = N then triggers the merging of features f1,f2.S --> A N IS BELOW A NS.tr = N.fS.lm = N.fS.rel = ‘below’N --> circleN.f = ‘circle’--> squareN.f = ‘square’The nonterminal merging step also renders the two original productions for S identical, and means theassociated feature equations should be reconciled. To this end, the two instances of ‘circle’ are implicitlyattributed to the first and second N.f feature (as part of the combined merging operators discussed in theprevious section).5.3.5 PAG PriorsPrior probability distributions for PAGs can be constructed using the now familiar techniques. Theprobabilities for the features equations (involving both fixed values and RHS features) represent multinomialparameter tuples for each LHS feature, and are accordingly covered by a Dirichlet prior distribution. Thefeature equations themselves can be given priors based on description length. Each equation involving aconstant feature value comes at a code length cost of log .h!. bits. Equations with features on the RHS are.f”)encoded using log is the set of featuresattached to nonterminal throughout the grammar (since only those need to be encoded uniquely), and"isthe length of production RHS.. N log"bits, where ) is the RHS nonterminal in question,f”Using a description length prior favors feature merging and other operations that achieve morecompact feature specifications by virtue of collapsing feature names and equations. Alternatively, one couldmake all feature specifications a priori equally likely 4 and rely solely on the prior of the context-free part ofthe grammar for a bias for smaller rules. Since feature equations cannot exist independently of context-freeproductions this will implicitly also bias the feature descriptions against complexity. Furthermore, redundant4 This is not strictly possible for a proper prior since the probability mass has to sum to one, so there has to be some sort of decayor cut-off for large grammars. However, when comparing grammars of limited, roughly equal size we can use a uniform prior as anapproximation.