Notes on computational linguistics.pdf - UCLA Department of ...

Stabler - Lx 185/209 2003
8.2.2 Stochastic CKY parser
(133) We have extended the CKY parsing strategy can handle any CFG, and augmented the chart entries so
that they indicate the rule used to generate each item and the positions of internal boundaries.
We still have a problem getting the parses out of the chart, since there can be too many of them: we do
not want to take out one at a time!
Onethingwecandoistoextractjustthemostprobableparse. Anequivalentideaistomakeallthe
relevant comparisons before adding an item to the chart.
(134) For any input string, the CKY parser chart represents a grammar that generates only the input string. We
can find the most probable parse using the Trellis-like construction familiar from Viterbi’s algorithm.
(135) For any positions 0 ≤ i, j ≤|input|, wecanfindtherule(i, j) : A : X with maximal probability.
(i − 1,ai,i) [axiom]
(i,a,j)
(i,A,j,p)
(i,B,j,p1) (j,C,k,p2)
(i,A,k,p1 ∗ p2 ∗ p3)
[r educe1] if A p → a
[r educe2] if A p3
→ BC
and ¬∃A p′
→ a such that p ′ >p
and ¬∃(i, B ′ ,j,p ′ 1 ),
(j, C ′ ,k,p ′ 2 ),
A p′ 3
→ B ′ C ′ such that
p ′ 1 ∗ p′ 2 ∗ p′ 3 >p1 ∗ p2 ∗ p3
(136) This algorithm does (approximately) as many comparisons of items as the non-probabilistic version,
since the reduce rules require identifying the most probable items of each category over each span of
the input.
To reduce the chart size, we need to restrict the rules so that we do not get all the items in there – and
then there is a risk of missing some analyses.
161