PhD thesis - School of Informatics - University of Edinburgh

More documents

Recommendations

Info

Chapter 1 Introduction Die Gewalt der Sprache ist nicht, dass sie das Fremde abweist, sondern dass sie es verschlingt. The power of language is not that it rejects foreign elements but that it devours them. JOHANN WOLFGANG VON GOETHE Geographic language mixing is a well known phenomenon that occurs on the borders between countries, where two different languages borrow useful words, phrases or sayings from each other. However, the growth of Internet has radically changed this once localised phenomenon into virtual language mixing, as it sweeps away all physical restrictions once imposed by borders. There are now practically no limits to language mixing as an Internet page can contain text in many different languages. Given that the majority of text published on the Internet is in English, an ever increasing number of its expressions and names are appearing even in once geographically distant languages. A good example of this phenomenon is German where the number of anglicisms in the language has increased considerably in recent years (e.g. Yang, 1990; Schütte, 1996; Utzig, 2002; Androutsopoulos et al., 2004; Tomaschett, 2005). There is also strong evidence for the increasing influence of English in many other languages such as French, Chinese, Japanese, Korean, Hindi and Russian. Even so, the intensity of English infiltration, as well as the impetus behind this process, can vary significantly between these languages. It has therefore become a focal point for con- 1
Chapter 1. Introduction 2 siderable research over the last few years (e.g. Kupper, 2003; Hall-Lew, 2002; Moody, 2006; Yoneoka, 2005; Kachru, 2006; Dunn, 2007). In fact, entire conferences are now dedicated to the subject area (e.g. the international conference on Anglicisms in Europe (2006) and the workshop on Strategies of Integrating and Isolating Non-native Entities and Structures to be held in February 2008). Any form of language mixing presents a source of considerable difficulty in nat- ural language processing (NLP) applications that automatically parse data, perform text-to-speech synthesis or translate data between languages. Current tools make the assumption that there is no language mixing, considering any input text to be mono- lingual, and consequently fail to process foreign language inclusions correctly. In ma- chine translation, this failure could result in anything from direct transfer of the foreign word into processed text to a complete loss of meaning through language confusion. Linguists who study the language mixing phenomenon normally rely on painstaking manual analysis of data to draw conclusions about the occurrence of anglicisms in any given language. This is not only time-consuming and cumbersome, but also date spe- cific. The reason for this is that languages evolve over relatively short timescales; the point at which collection of data for a corpus begins will almost certainly have different language mixing characteristics from the point at which it ends, especially if the corpus is collected over a few years. This underlines a real need for new automatic foreign language inclusion detection techniques. Increased availability of electronic data in different languages has encouraged the NLP research community to devote greater attention to multilingual and language in- dependent applications. However, the task of detecting foreign words and names in mixed-lingual text remains beyond the capabilities of existing automatic language identification techniques (e.g. Beesley, 1988; Dunning, 1994; Cavnar and Trenkle, 1994; Damashek, 1995; Ahmed et al., 2004) which only really perform well when identifying the base language of a sentence, paragraph or document. The problem with such existing techniques is that they are based on character language models or n-gram frequencies, both of which statistically analyse typical character sequences in languages. Unfortunately, this means that they are not suited to language identification at the word level and therefore do not deal well with mixed-lingual text. Only a few research groups are active in the field of mixed-lingual language identification (Pfis- ter and Romsdorfer, 2003; Marcadet et al., 2005; Farrugia, 2005; Andersen, 2005, all
Page 1 and 2: Automatic Detection of English Incl
Page 3 and 4: these parsers with the annotation-f
Page 5 and 6: Declaration I declare that this the
Page 7 and 8: 3.3.5 Post-processing Module . . .
Page 9 and 10: A.2.2 Kappa Coefficient . . . . . .
Page 11 and 12: 5.6 Average relative token frequenc
Page 13: 3.16 Most frequent English inclusio
Page 17 and 18: Chapter 1. Introduction 4 Chapter 3
Page 19 and 20: Chapter 1. Introduction 6 1.1 Relat
Page 21 and 22: Chapter 2. Background and Theory 8
Page 59 and 60: Chapter 3 Tracking English Inclusio
Page 61 and 62: Chapter 3. Tracking English Inclusi
Page 63 and 64: Chapter 3. Tracking English Inclusi
Page 65 and 66:
Chapter 3. Tracking English Inclusi
Page 67 and 68:
Page 69 and 70:
Page 71 and 72:
Page 73 and 74:
Page 75 and 76:
Page 77 and 78:
Page 79 and 80:
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Chapter 4 System Extension to a New
Page 115 and 116:
Chapter 4. System Extension to a Ne
Page 117 and 118:
Page 119 and 120:
Page 121 and 122:
Page 123 and 124:
Page 125 and 126:
Page 127 and 128:
Page 129 and 130:
Chapter 5 Parsing English Inclusion
Page 131 and 132:
Chapter 5. Parsing English Inclusio
Page 133 and 134:
Page 135 and 136:
Page 137 and 138:
Page 139 and 140:
Page 141 and 142:
Page 143 and 144:
Page 145 and 146:
Page 147 and 148:
Page 149 and 150:
Page 151 and 152:
Page 153 and 154:
Page 155 and 156:
Page 157 and 158:
Page 159 and 160:
Chapter 6 Other Potential Applicati
Page 161 and 162:
Chapter 6. Other Potential Applicat
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185 and 186:
Page 187 and 188:
Chapter 7 Conclusions and Future Wo
Page 189 and 190:
Chapter 7. Conclusions and Future W
Page 191 and 192:
Appendix A. Evaluation Metrics and
Page 193 and 194:
Page 195 and 196:
Page 197 and 198:
Page 199 and 200:
Appendix B. Guidelines for Annotati
Page 201 and 202:
Page 203 and 204:
Page 205 and 206:
Appendix C TIGER Tags and Labels C.
Page 207 and 208:
Appendix C. TIGER Tags and Labels 1
Page 209 and 210:
Appendix C. TIGER Tags and Labels 1
Page 211 and 212:
Bibliography 198 Andersen, G. (2005
Page 213 and 214:
Bibliography 200 Bresnan, J. (2001)
Page 215 and 216:
Bibliography 202 Damashek, M. (1995
Page 217 and 218:
Bibliography 204 Finkel, J., Dingar
Page 219 and 220:
Bibliography 206 Hachey, B., Alex,
Page 221 and 222:
Bibliography 208 Kirkness, A. (1984
Page 223 and 224:
Bibliography 210 and Technology (In
Page 225 and 226:
Bibliography 212 Poplack, S. (1988)
Page 227 and 228:
Bibliography 214 Sokol, D. K. (2000
Page 229:
Bibliography 216 Yang, W. (1990). A
show all

PhD thesis - School of Informatics - University of Edinburgh

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?