Foundations of Data Science

previous state j by putting an arrow with edge label t from i to j. At the end, can find
the most likely sequence by tracing backwards as is standard for dynamic programming
algorithms.
Example: For the earlier example what is the most likely sequence of states to produce
the output hhht?
t = 3 max{ 1
t = 2 max{ 1 8
t = 1
1
1 1
2 2
1 1
, 1 3 1
48 2 2 24 4
1 1
, 1 3 1
2 2 6 4
} = 1 q or p max{ 3
2 64
} = 3 p max{ 1 2 48 8
= 1 1
q
2 8 2
1 2
2
t = 0
1
2
q 0 p
q
1 1
, 1 1 1
48 2 3 24 4
1 2
, 1 1 2
2 3 6 4
3 = 1 6
q
3 } = 1
3 } = 1
p
96
q
24
q
Note that the two sequences of states, qqpq and qpqq, are tied for the most likely sequences
of states.
Determining the underlying hidden Markov model
Given an n-state HMM, how do we adjust the transition probabilities and output
probabilities to maximize the probability of an output sequence O 1 O 2 · · · O T ? The assumption
is that T is much larger than n. There is no known computationally efficient
method for solving this problem. However, there are iterative techniques that converge
to a local optimum.
Let a ij be the transition probability from state i to state j and let b j (O k ) be the
probability of output O k given that the HMM is in state j. Given estimates for the HMM
parameters, a ij and b j , and the output sequence O, we can improve the estimates by
calculating for each unit of time the probability that the HMM goes from state i to state
j and outputs the symbol O k .
Given estimates for the HMM parameters, a ij and b j , and the output sequence O, the
probability δ t (i, j) of going from state i to state j at time t is given by the probability of
producing the output sequence O and going from state i to state j at time t divided by
the probability of producing the output sequence O.
δ t (i, j) = a t(i)a ij b j (O t+1 )β t+1 (j)
p(O)
The probability p(O) is the sum over all pairs of states i and j of the numerator in the
above formula for δ t (i, j). That is,
p(O) = ∑ ∑
α t (j)a ij b j (O t+1 )β t+1 (j).
i j
306