Notes on computational linguistics.pdf - UCLA Department of ...

Stabler - Lx 185/209 2003
c. Finally, given P(qi ⇒ a1 ...an) for all qi ∈ ΩX,
P(a1 ...an) =
P0(qi)P(qi ⇒ a1 ...an)
qi∈ΩX
(86) Exercise: Use the coffee machine as elaborated in (84) and the backward method to compute the probability
of the output sequence
s1s3s3.
8.1.11 Computing most probable parses: Viterbi’s algorithm
(87) Given a string a1 ...an output by a Markov model, what is the most likely sequence of states that could
have yielded this string? This is analogous to finding a most probable parse of a string.
Notice that we could solve this problem by calculating the probabilities of the output sequence for each
of the |ΩX| n state sequences, but this is not feasible!
(88) The Viterbi algorithm allows efficient calculation of the most probable sequence of states producing
a given output (Viterbi 1967; Forney 1973), using an idea that is similar to the forward calculation of
output sequence probabilities in §8.1.9 above.
Intuitively, once we know the best way to get to any state in ΩX at a time t, the best path to the next
state is an extension of one of those.
(89) a. Calculate, for each possible initial state qi ∈ ΩX,
P(qi,a1) = P0(qi)P(a1|qi).
and record: qi : P(qi,a1)@ɛ.
That is, for each state qi, we record the probability of the state sequence ending in qi.
b. Recursive step: Given qi : P(qqi,a1 ...at)@q for each qi ∈ ΩX,
for each qj ∈ ΩX find a qi that maximizes
P(qqiqj,a1 ...atat+1) = P(qqi,a1 ...at)P(qj|qi)P(at+1|qj)
and record: qj : P(qqiqj,a1 ...atat+1)@qqi. 37
c. After these values have been computed up to the final state tn, we choose a qi : P(qqi,a1 ...an)@q
with a maximum probability P(qqi,a1 ...an).
(90) Exercise: Use the coffee machine as elaborated in (84) to compute the most likely state sequence underlying
the output sequence
s1s3s3.
(91) The Viterbi algorithm is not incremental: at every time step |ΩX| different parses are being considered.
As stated, the algorithm stores arbitrarily long state paths at each step, but notice that each step only
needs the results of the previous step: |ΩX| different probabilities (an unbounded memory requirement,
unless precision can be bounded)
37In case more than one qi ties for the maximum P(qqiqj,a1 ...atat+1), we can either make a choice, or else carry all the winning
options forward.
146