Notes on computational linguistics.pdf - UCLA Department of ...

Stabler - Lx 185/209 2003
a. probability of the first symbol s1 from one of the initial states
p(qi|s1) = p(qi)p(s1|qi) = 0.7 · 0.8
0.2 · 0.1
0.1 · 0.2
= 0.56 0.02 0.02
b. probabilities of the following symbols from each state (transposed to column matrix)
p(qi|s1s3) ′ ⎡
⎤
((p(q1,s1) · p(q1|q1)) + (p(q2,s1) · p(q1|q2)) + (p(q3,s1) · p(q1|q3))) · p(s3|q1)
⎢
⎥
= ⎣((p(q1,s1)
· p(q2|q1)) + (p(q2,s1) · p(q2|q2)) + (p(q3,s1) · p(q2|q3))) · p(s3|q2) ⎦
⎡
((p(q1,s1) · p(q3|q1)) + (p(q2,s1) · p(q3|q2))
⎤
+ (p(q3,s1) · p(q3|q3))) · p(s3|q3)
((0.56 · 0.7) + (0.02 · 0.2) + (0.02 · 0)) · 0.1
⎢
⎥
= ⎣((0.56
· 0.2) + (0.02 · 0.7) + (0.02 · 0)) · 0.1⎦
⎡
((0.56 · 0.1) + (0.02 · 0.1) +
⎤
(0.02 · 1)) · 0.6
(0.392 + 0.04) · 0.1
⎢
⎥
= ⎣ (0.112 + 0.014) · 0.1 ⎦
⎡
(0.056
⎤
+ 0.002 + 0.02) · 0.6
0.0432
⎢ ⎥
= ⎣0.0126⎦
0.0456
p(qi|s1s3s3) ′ ⎡
⎤
((p(q1,s1s3) · p(q1|q1)) + (p(q2,s1s3) · p(q1|q2)) + (p(q3,s1s3) · p(q1|q3))) · p(s3|q1)
⎢
⎥
= ⎣((p(q1,s1s3)
· p(q2|q1)) + (p(q2,s1s3) · p(q2|q2)) + (p(q3,s1s3) · p(q2|q3))) · p(s3|q2) ⎦
⎡
((p(q1,s1s2) · p(q3|q1)) + (p(q2,s1s3) · p(q3|q2)) +
⎤
(p(q3,s1s3) · p(q3|q3))) · p(s3|q3)
((0.0432 · 0.7) + (0.0126 · 0.2) + (0.0456 · 0)) · 0.1
⎢
⎥
= ⎣((0.0432
· 0.2) + (0.0126 · 0.7) + (0.0456 · 0)) · 0.1⎦
⎡
((0.0432 · 0.1) + (0.0126 · 0.1) + (0.0456
⎤
· 1)) · 0.6
(0.03024 + 0.00252) · 0.1
⎢
⎥
= ⎣ (0.00864 + 0.00882) · 0.1 ⎦
⎡
(0.00432
⎤
+ 0.00126 + 0.0456) · 0.6
0.003276
⎢ ⎥
= ⎣0.001746⎦
0.030708
c. Finally, we calculate p(s1s3s3) as the sum of the elements of the last matrix:
p(s1s3s3) = 0.03285
8.1.10 Computing output sequence probabilities: backward
Another feasible way to compute the probability of an output sequence a1 ...an.
(85) a. Let P(qi ⇒ a1 ...an) be the probability of emitting a1 ...an beginning from state qi.
And for each possible final state qi ∈ ΩX, let
P(qi ⇒ ɛ) = 1
b.
(With this base case, the first use of the recursive step calculates P(qj ⇒ an) for each qi ∈ ΩX.)
Recursive step: Given P(qi ⇒ at ...an) for all qi ∈ ΩX, calculateP(qj ⇒ at−1 ...an) for all qj ∈ ΩX
as follows:
P(qj ⇒ at−1 ...an) =
P(qi ⇒ at ...an)P(qj|qi) P(at−1|qj)
j∈ΩX
145