The dissertation of Andreas Stolcke is approved: University of ...

2. For each candidate"I!computeLet"$ 3. be the merge that maximizes +9,",4@@@4@the merged model",the samples ) in . These states are chained with transitions of probability 1, such that a sample 1 £££1 ”•4@@4@@@4,@@CHAPTER 3. HIDDEN MARKOV MODELS 40A. Get some new samples )and incorporate into the current model 4.B. Loop:1. Compute a set of candidate merges! from among the states of model 4.0. )g0 .0 and its posterior probability+-,",0. )g0 . Then let 4#1 :6#">0 .5. Let H :6 HEN 1.+9,# 1. )g0ŒÏ4. If +-, 4. )g0 , break from the loop.C. If the data is exhausted, break from the loop and 4 return as the induced model.Incremental merging might in principle produce results worse than the batch version since theevaluation step doesn’t have as much data at its disposal. However, we didn’t find this to be a significantdisadvantage in practice. One can optimize the number of samples incorporated in each step A (the batch size)for overall speed. This requires balancing the gains due to smaller model size against the constant overhead ofeach execution of step B. The best value will depend on the data and how much merging is actually possibleon each iteration; we found between 1 and 10 samples at a time to be good choices.One has to be careful not to start merging with extremely small models, such as that resulting fromincorporating only a few short samples. Many of the priors discussed earlier contain logarithmic terms thatapproach singularities (log 0) in this case, which can produce poor results, usually by leading to extrememerging. That can easily be prevented by incorporating a larger number of samples (say, 10 to 20) beforegoing on to the first merging step.Further modifications to the simple best-first search strategy are discussed in Section 3.4.5.3.4 Implementation IssuesIn this section we elaborate on the implementation of the various steps in the generic HMM mergingalgorithm presented in Section 3.3.5.3.4.1 Efficient sample incorporationIn the simplest case this step creates a dedicated state for each instance of a symbol in any ofÔ 1is generated by a 1£££ u Ôstateusequence . 1 can be reached from via a transition of probability 1· u u. where is the total number ofÔsamples. State connects to with probability 1. All states u @)/.corresponding output symbol with probability 1.@ ½ u 1 u, •emit their