Foundations of Data Science

for t = 1 to T
Prob(O 0 O 1 · · · O t , i) = ∑ j
Prob(O 0 O 1 · · · O t−1 , j)a ij p(O t+1 , i)
Example: What is the probability of the sequence hhht by the HMM in the above two
state example?
t = 3
3
t = 2
1
t = 1
1
1 1
+ 5 3 1
= 19
32 2 2 72 4 2 384
1 1
+ 1 3 1
= 3
8 2 2 6 4 2 32
1 1
= 1
2 2 2 8
t = 0
1
2
0
q
3 1 1
+ 5 1 1
= 37
32 2 3 72 4 3 64×27
1 1 2
+ 1 1 2
= 5
8 2 3 6 4 3 72
1 1 2
= 1
2 2 3 6
For t = 0, the q entry is 1/2 since the probability of being in state q is one and the probability
of outputting heads is 1 . The entry for p is zero since the probability of starting in
2
state p is zero. For t = 1, the q entry is 1 since for t = 0 the q entry is 1 and in state q
8 2
the HMM goes to state q with probability 1 and outputs heads with probability 1. The
2 2
p entry is 1 since for t = 0 the q entry is 1 and in state q the HMM goes to state p with
6 2
probability 1 and outputs heads with probability 2 3
. For t = 2, the q entry is which
2 3 32
consists of two terms. The first term is the probability of ending in state q at t = 1 times
the probability of staying in q and outputting h. The second is the probability of ending
in state p at t = 1 times the probability of going from state p to state q and outputting h.
From the table, the probability of producing the sequence hhht is 19
384 + 37
1728 = 0.0709.
p
The most likely sequence of states - the Viterbi algorithm
Given a HMM and an observation O = O 0 O 1 · · · O T , what is the most likely sequence of
states? The solution is given by the Viterbi algorithm, which is a slight modification to the
dynamic programming algorithm just given for determining the probability of an output
sequence. For t = 0, 1, 2, . . . , T and for each state i, calculate the probability of the most
likely sequence of states to produce the output O 0 O 1 O 2 · · · O t ending in state i. For each
value of t, calculate the most likely sequence of states by selecting over all states j the most
likely sequence producing O 0 O 1 O 2 · · · O t and ending in state i consisting of the most likely
sequence producing O 0 O 1 O 2 · · · O t−1 ending in state j followed by the transition from j
to i producing O t . Note that in the previous example, we added the probabilities of each
possibility together. Now we take the maximum and also record where the maximum
came from. The time complexity is O(n 2 T ) and the space complexity is O(nT ). The
space complexity bound is argued as follows. In calculating the probability of the most
likely sequence of states that produces O 0 O 1 . . . O t ending in state i, we remember the
305