A Treebank-based Investigation of IPP-triggering Verbs in Dutch

to parse now does. The extra manual annotation effort in treebank update is relatively
small when compared to the first round of annotation, typically requiring
one or two additional decisions for some 5–10% of the previously recorded correct
analyses, and new annotation for previously rejected items, which were another
15% of the total in the second round, and much less in successive rounds. Hence
these later rounds of updating the treebank proceed more quickly than the initial
round of annotation.
Correcting errors of both classes based on analysis of the first pass through
DeepBank annotation has resulted in a significant improvement in coverage and
accuracy for the ERG over the WSJ corpus. Raw coverage has risen by some 10%
from the first pass and the ‘survival’ rate of successfully treebanked sentences has
risen even more dramatically to more than 80% of all sentences in the first 16
sections of the WSJ that have now gone through two rounds of grammar/treebank
updates. The table below shows the current status of these first 16 sections of
the English DeepBank in terms of “Observed” and “Verified” coverage, where the
former reports the number of sentences that received at least one analysis from the
ERG, and the latter gives the number of sentences for which the annotator recorded
a correct analysis.
Table 1: English DeepBank ERG results for WSJ Sections 00–15
Section Number of items Observed coverage Verified coverage
00 1922 92.2% 82.0%
01 1997 92.3% 81.6%
02 1996 92.3% 84.0%
03 1482 92.0% 82.1%
04 2269 92.6% 81.5%
05 2137 92.3% 81.8%
06 1835 91.3% 81.1%
07 2166 91.9% 82.6%
08 478 90.6% 80.1%
09 2073 92.0% 81.2%
10 1945 91.8% 81.3%
11 2237 91.5% 80.4%
12 2124 94.2% 85.1%
13 2481 94.8% 85.8%
14 2182 94.0% 86.0%
15 2118 94.1% 86.4%
Subtotal 31442 92.6% 82.6%
These figures, which are surprisingly stable across sections both in raw parsing
coverage and in treebanked items, show that roughly 18% of the sentences in the
corpus fail to receive a correct analysis from the ERG; we discuss the DeepBank
annotations for this portion of the corpus in section 4. Note that most of the remaining
portion of the WSJ corpus has now been treebanked the first time through,
and we expect the remaining updated sections to be completed by the end of the
year, excluding three held-out sections reserved for future testing.
89