The dissertation of Andreas Stolcke is approved: University of ...

,CHAPTER 3. HIDDEN MARKOV MODELS 56b0.170.47a0.48c0.440.430.24 0.02 0.83 0.290.240.30 0.26Start0.060.03a0.98c0.530.190.760.04a0.221.00b0.43c0.81c0.78b1.00EndFigure 3.5: Case study I: Redundant BW-derived HMM structure for " .Baum-Welch studies It is instructive to inspect some of the HMM topologies found by the Baum-Welchestimator. Figure 3.4 shows models of 6 states trained on minimal samples, one exhibiting overgeneralization,and one demonstrating both overfitting and overgeneralization.The HMM in (a) generates , !"07 !K0 and has redundantly allocated states to generate !" .The HMM in (b) generates , ëT 0776,0 1 2 3. Here, precious states have been wasted, for"/6 -T0modeling the repetition of ’s, instead of generalizing to a loop over a single state and using those states tomodel the distinction between and .If estimation using the minimal number of states (6 in this case) is successful, the discretizedstructure invariably is that of the target model (Figure 3.2), as expected, although the probabilities will dependon the training sample used. Successful induction using 10 states, on the other hand, leads to models that,by definition, contain redundant states. However, the redundancy is not necessarily a simple duplication ofstates found in the target model structure. Instead, rather convoluted structures are found, such as the one inFigure 3.5 (induced from the random 20 samples).Merging studies We also investigated how the merging algorithm behaves for non-optimal values of theglobal prior weight..As explained earlier, this value is implicit in the number of ‘effective’ samples, theparameter that was maintained constant in all experiments, and which seems to be robust over roughly anorder of magnitude.We therefore took resulting. the value and adjusted it both upward and downward by an order of