The dissertation of Andreas Stolcke is approved: University of ...

,CHAPTER 3. HIDDEN MARKOV MODELS 60(a)0.26a1.000.210.570.650.530.510.350.270.00Start0.00a0.43b0.23a bEnd(b)0.270.00 0.210.39 0.250.570.000.471.000.520.610.43Start ab a b0.28EndFigure 3.9: Case study II: BW-derived HMM structures that fail on generalization.Baum-Welch studies As in the previous case study, we looked at various model structures found byBaum-Welch estimation. All examples in this section are from training on the 20 random samples.Figure 3.9(a) shows a structure that is overly general: it generates , %§E#&#%0have an HMM that partly overgeneralizes, but at the same time exhibits a rather peculiar case of overfitting: itw¤07#& # . In (b), weexcludes strings of the >#&6#&# form where"is even. No such cases happened to be present in the trainingset.The accurate model structures of 10 states found by the Baum-Welch method again tended to berather convoluted. Figure 3.10 shows as case in point.Merging studies We also repeated the experiment examining the levels of generalization by the mergingalgorithm, as the value of the global prior weight was increased over three orders of magnitude.Figure 3.11 shows the progression of models for. 6 0£ 018 0£ 18, and 1£ 0. The pattern is similar tothat in in the first case study (Figure 3.11). The resulting models range from a simple merged representation ofthe samples to a plausible overgeneralization from the training data ( , B#&#%07# ). The target model is obtainedfor.values between these two extremes.