PhD thesis - School of Informatics - University of Edinburgh

Chapter 5. Parsing English Inclusions 133
Phrase bracket (PB) frequency BL IE
PBPRED > PBGOLD 62% 51%
PBPRED < PBGOLD 11% 13%
PBPRED = PBGOLD 27% 36%
Table 5.6: Bracket frequency of the predicted baseline (BL) and the inclusion entity (IE)
model output compared to the gold standard.
This analysis suggests that the annotation guidelines on foreign inclusions could be
improved when differentiating between phrase categories containing foreign material.
Despite the few inconsistencies and annotation errors discussed here, the large major-
ity of English inclusions is consistently annotated as either PN or CH phrase. In the
following, the errors in the parsing output of the inclusion set are examined in detail.
5.3.5.2 Phrase Bracketing
Table 5.6 summarises the number of phrase bracketing errors made for the inclusion
set. For the majority of sentences (62%), the baseline model predicts more brackets
than are present in the gold standard parse tree. This number decreases by 11% to
51% when parsing with the inclusion entity model. The baseline parser predicts fewer
phrase brackets in the output compared to the gold standard in only 11% of sentences.
This number slightly increases to 13% for the inclusion entity model. Generally, these
numbers suggest that the baseline parser does not recognise English inclusions as con-
stituents and instead parses their individual tokens as separate phrases. Provided with
additional information of multi-word English inclusions in the training data, the parser
is able to overcome this problem. This conclusion is further substantiated in the next
section which examines parsing errors specifically caused by English inclusions.
5.3.5.3 Parsing Errors
In order to understand the parser’s treatment of English inclusions, each parse tree is
analysed as to how accurate the baseline and inclusion entity models are at predict-
ing both phrase bracketing and phrase categories (see Table 5.7). For 46 inclusions
(42.2%), the baseline parser makes an error with a negative effect on performance.