Notes on computational linguistics.pdf - UCLA Department of ...

Stabler - Lx 185/209 2003
P(q1 ...qn) = P0(q1)
1≤i≤n−1
Given q1 ...qn, the probability of output sequence a1 ...an is
P(qi+1|qi) (i)
n
P(at|qt). (ii)
t=1
The probability of q1 ...qn occurring with outputs a1 ...an is the product of the two probabilities (i)
and (ii), that is,
P(q1 ...qn,a1 ...an) = P0(q1)
1≤i≤n−1
P(qi+1|qi)
n
P(at|qt). (iii)
(81) Given any Markov model, the probability of output sequence a1 ...an is the sum of the probabilities of
this output for all the possible sequences of n states.
P(q1 ...qn,a1 ...an) (iv)
qi∈ΩX
(82) Directly calculating this is infeasible, since there are |ΩX| n state sequences of length n.
8.1.9 Computing output sequence probabilities: forward
Here is a feasible way to compute the probability of an output sequence a1 ...an.
(83) a. Calculate, for each possible initial state qi ∈ ΩX,
P(qi,a1) = P0(qi)P(a1|qi).
b. Recursive step: Given P(qi,a1 ...at) for all qi ∈ ΩX, calculateP(qj,a1 ...at+1) for all qj ∈ ΩX as
follows
P(qj,a1 ...at+1) =
P(qi,a1 ...at)P(qj|qi) P(at+1|qj)
i∈ΩX
c. Finally, given P(qi,a1 ...an) for all qi ∈ ΩX,
P(a1 ...an) =
qi∈ΩX
P(qi,a1 ...an)
(84) Let’s develop the coffee machine example from (66), adding outputs so that we have a Markov model
instead of just a Markov chain. Suppose that there are 3 output messages:
(s1) thank you
(s2) no change
(s3) x@b*/!
Assume that these outputs occur with the probabilities given in the following matrix where row i column
j represents the probability of emitting symbol sj ⎡
when
⎤
in state i:
0.8 0.1 0.1
⎢
⎥
O= ⎣0.1
0.8 0.1⎦
0.2 0.2 0.6
Exercise: what is the probability of the output sequence
s1s3s3
Solution sketch: (do it yourself first! note the trellis-like construction)
144
t=1