PhD thesis - School of Informatics - University of Edinburgh

More documents

Recommendations

Info

Chapter 3. Tracking English Inclusions in German 85 occurring in advertising are conspicuous for nearly all of the respondents (98%). A closer look at the data reveals that Marcadet et al. (2005) do not consistently annotate abbreviations and acronyms expanding to English definitions as English. Conversely, the English inclusion classifier presented in this thesis is designed to recognise them as well. Moreover, person names like Ted Saskins are annotated as English and not distinguished from real English inclusions as advocated in this thesis. On the reconciled gold standard, the English inclusion classifier performs with an F-score of 96.35 points (an accuracy of 98.95%, a precision of 97.78% and a recall of 94.96%). These scores are slightly better than those reported by Marcadet et al. (2005) on this data set (98.67% accuracy). However, it is not entirely straightforward to com- pare these scores as the gold standard annotation is reconciled. The few classification errors are mainly due to English words like Team or Management being already listed in the German lexicon. These anglicisms are strongly integrated in the German language and have well established pronunciations. Therefore, such classification errors are unlikely to cause pronunciation problems during TTS synthesis. Given the results of both sets of evaluations, it can be concluded that the English inclusion classifier performs well on randomly selected unseen mixed-lingual data in different domains and compares well to an existing mixed-lingual LID approach. 3.5 Parameter Tuning Experiments This section discusses a series of interesting parameter tuning experiments to optimise the English inclusion classifier. These were the basis for the final design of the full system which was evaluated in the previous section. These experiments include a task- based evaluation of three different POS taggers and a task-based evaluation of two search engines. All experiments involve the German development data for evaluation. 3.5.1 Task-based Evaluation of Different POS taggers Throughout the entire process of error analysis, it was noticed that the performance of the English inclusion classifier depends to some extent on the performance on the POS tagger. Initially, the system made use of the POS tagger TnT (Brants, 2000b) trained on the NEGRA corpus (Skut et al., 1997). Some classification errors result from errors
Chapter 3. Tracking English Inclusions in German 86 made by the POS tagger and therefore could be avoided if the latter performed with perfect accuracy. One reason for lower tagging accuracy is the fact that POS taggers trained on data for a particular language do not necessarily deal well with text containing foreign inclusions. Moreover, some taggers have difficulty differentiating between common and proper nouns in some cases. In order to gain a better understanding of how the POS tagging influences the performance of the English inclusion classifier, I compared three different taggers in a task-based evaluation: • TnTNEGRA - the TnT tagger trained on the NEGRA corpus of approximately 355,000 tokens (Skut et al., 1997) • TnTTIGER - the TnT tagger trained on the TIGER corpus of approximately 700,000 tokens (Brants et al., 2002) • TreeTagger - the TreeTagger trained on a small German newspaper corpus of Stuttgarter Zeitung containing 25,000 tokens (Schmid, 1994, 1995) The English inclusion classifier is essentially run on the same set of data tagged by the three different POS taggers and evaluated against the hand-annotated gold standard. Note that this method does not necessarily determine the best and most accurate POS tagger but rather one that is most beneficial for identifying English inclusions in German text. Before discussing the results for each setup, the characteristics of the two POS taggers used in this evaluation are explained in detail. 3.5.1.1 TnT - Trigrams’n’Tags TnT is a very efficient statistical POS tagger which can be trained on corpora in different languages and domains and new POS tag sets (Brants, 2000b). Moreover, the tagger is very fast to train and run. It is based on the Viterbi algorithm for second order Markov models and therefore assigns the tag ti that is most likely to generate the wi given the two previous tags ti−1 and ti−2. The output and transition probabili- ties are estimated from an annotated corpus. In order to deal with data sparseness, the system incorporates linear interpolation-based smoothing and handles unknown words via n-gram-based suffix analysis.
Page 1 and 2:
Automatic Detection of English Incl
Page 3 and 4:
these parsers with the annotation-f
Page 5 and 6:
Declaration I declare that this the
Page 7 and 8:
3.3.5 Post-processing Module . . .
Page 9 and 10:
A.2.2 Kappa Coefficient . . . . . .
Page 11 and 12:
5.6 Average relative token frequenc
Page 13 and 14:
3.16 Most frequent English inclusio
Page 15 and 16:
Chapter 1. Introduction 2 siderable
Page 17 and 18:
Chapter 1. Introduction 4 Chapter 3
Page 19 and 20:
Chapter 1. Introduction 6 1.1 Relat
Page 21 and 22:
Chapter 2. Background and Theory 8
Page 23 and 24:
Page 25 and 26:
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
Page 37 and 38:
Page 39 and 40:
Page 41 and 42:
Page 43 and 44:
Page 45 and 46:
Page 47 and 48: Chapter 2. Background and Theory 34
Page 59 and 60: Chapter 3 Tracking English Inclusio
Page 61 and 62: Chapter 3. Tracking English Inclusi
Page 97: Chapter 3. Tracking English Inclusi
Page 113 and 114: Chapter 4 System Extension to a New
Page 115 and 116: Chapter 4. System Extension to a Ne
Page 129 and 130: Chapter 5 Parsing English Inclusion
Page 131 and 132: Chapter 5. Parsing English Inclusio
Page 149 and 150:
Chapter 5. Parsing English Inclusio
Page 151 and 152:
Page 153 and 154:
Page 155 and 156:
Page 157 and 158:
Page 159 and 160:
Chapter 6 Other Potential Applicati
Page 161 and 162:
Chapter 6. Other Potential Applicat
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185 and 186:
Page 187 and 188:
Chapter 7 Conclusions and Future Wo
Page 189 and 190:
Chapter 7. Conclusions and Future W
Page 191 and 192:
Appendix A. Evaluation Metrics and
Page 193 and 194:
Page 195 and 196:
Page 197 and 198:
Page 199 and 200:
Appendix B. Guidelines for Annotati
Page 201 and 202:
Page 203 and 204:
Page 205 and 206:
Appendix C TIGER Tags and Labels C.
Page 207 and 208:
Appendix C. TIGER Tags and Labels 1
Page 209 and 210:
Appendix C. TIGER Tags and Labels 1
Page 211 and 212:
Bibliography 198 Andersen, G. (2005
Page 213 and 214:
Bibliography 200 Bresnan, J. (2001)
Page 215 and 216:
Bibliography 202 Damashek, M. (1995
Page 217 and 218:
Bibliography 204 Finkel, J., Dingar
Page 219 and 220:
Bibliography 206 Hachey, B., Alex,
Page 221 and 222:
Bibliography 208 Kirkness, A. (1984
Page 223 and 224:
Bibliography 210 and Technology (In
Page 225 and 226:
Bibliography 212 Poplack, S. (1988)
Page 227 and 228:
Bibliography 214 Sokol, D. K. (2000
Page 229:
Bibliography 216 Yang, W. (1990). A
show all

PhD thesis - School of Informatics - University of Edinburgh

Create successful ePaper yourself

Delete template?

Save as template?