himalayan linguistics - UCSB Linguistics - University of California ...

More documents

Recommendations

Info

Himalayan Linguistics, Vol 10(1) Palmer, F. R. 2001 (2nd edition). Mood and modality. [Cambridge Textbooks in Linguistics] Cambridge: Cambridge University Press. Sāymi, Dhūsvām̥ . VS 2026 [AD 1969–70] makha (upnyas) [Mother Hen (a novel)], Kantipur: Ratna Pustak Bhaɳɖhār. Shreʂʈa, Bhūʂaɳ Prasād, NS 1101 [AD 1980–81] kukulyãː kuː [Cock-a-doodle-doo], [Kathmandu] Jyaːbəhaː Lyæːmhə Pucəː Situ, Kedār. NS 1112 [AD 2000–2001] swəmhəmhə mənuː [The Third Person] Patanː Kīrti Saphū Kuthi. Austin Hale austin_hale@sall.com 150
Himalayan Linguistics, Vol. 10(1). © Himalayan Linguistics 2011 ISSN 1544-7502 Extending corpus annotation of Nepali: advances in tokenisation and lemmatisation Andrew Hardie Lancaster University, UK Ram R. Lohani Yogendra P. Yadava Tribhuvan University, Nepal 1 Introduction Nepali is an Indo-Aryan language spoken by approximately 45 million people in Nepal – where it is the language of government and the medium of much education – and in neighbouring countries (India, Bhutan, and Myanmar). It serves as the lingua franca of an extremely multilingual area of the world; more than 90 languages are spoken within Nepal. 1 Nepali is written in the Devanagari alphabet and has a written tradition extending back to the twelfth century CE. Within the last few years, corpus resources have been developed for Nepali for the first time in the form of the Nepali National Corpus (NNC; see Yadava et al., 2008), developed by the Bhasha Sanchar (“language communication”) project, also known as Nelralec. 2 A variety of Nepali language technology support projects were undertaken within Nelralec, including software localisation and font development. This process also created several resources for automated annotation of linguistic analyses of texts and corpora, most notably part-of-speech (POS) tagging. These analytic procedures were applied to, and are available within, the released version of the NNC. In this paper, we describe subsequent work to develop advances on the corpus annotation present in the NNC, with a focus on two forms of analysis in particular – namely, tokenisation (the splitting of text into word-level units of analysis) and lemmatisation (the assignment of every token to a “dictionary headword” or lemma). After giving an overview of the context of this work (§2), namely the design and composition of the NNC, we will discuss points of the original word-level annotation in the corpus which represent sub-optimal compromises made for practical reasons to address challenging aspects of morphology (§3) – compromises which, we would argue, are no longer necessary in light of the advances in lemmatisation and tokenisation which we present here. We will detail the mechanics of these advances, discussing the software used and the linguistic knowledge resources on which they rely, for tokenisation (§4) and lemmatisation (§5) respectively. In both cases, the software requires 1 According to the 2001 census of Nepal (CBS, 2001). 2 Nelralec: Nepali Language Resources and Localization for Education and Communication; funded by the EU Asia IT&C programme, reference number ASIE/2004/091-777. See http://www.bhashasanchar.org
Page 1 and 2:
volume 10 number 1 June 2011 himala
Page 3:
Himalayan Linguistics Volume 10, Nu
Page 7:
Michael Noonan in Nepal, sitting wi
Page 10 and 11:
Himalayan Linguistics, Vol 10(1) Fi
Page 12 and 13:
Himalayan Linguistics, Vol 10(1) Mi
Page 14 and 15:
Himalayan Linguistics, Vol 10(1) Re
Page 16 and 17:
Himalayan Linguistics, Vol 10(1) 20
Page 18 and 19:
Page 21 and 22:
Works by David Watters Books Forthc
Page 23:
xxi Works by David Watters 2006.
Page 26 and 27:
Himalayan Linguistics, Vol 10(1) La
Page 28 and 29:
Himalayan Linguistics, Vol 10(1) Th
Page 30 and 31:
Himalayan Linguistics, Vol 10(1) Is
Page 33 and 34:
Himalayan Linguistics, Vol. 10(1).
Page 35 and 36:
DeLancey: Verb agreement prefixes i
Page 37 and 38:
Page 39 and 40:
(6) amh bəyʔ-ne-wʔ food give-NON
Page 41 and 42:
Singular Plural 1 maʔ-niŋ maʔ-nu
Page 43 and 44:
(20) mín-rhê-reŋ-áŋ-cê 1OBJ
Page 45 and 46:
Page 47 and 48:
Page 49 and 50:
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
Page 59 and 60:
Page 61:
Page 64 and 65:
Himalayan Linguistics, Vol 10(1) wh
Page 66 and 67:
Himalayan Linguistics, Vol 10(1) fo
Page 68 and 69:
Himalayan Linguistics, Vol 10(1) un
Page 70 and 71:
Page 73 and 74:
Page 75 and 76:
43 Eppele: Language use among the B
Page 77 and 78:
Figure 2. Language 9 areas by VDC 4
Page 79 and 80:
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Genetti: Direct speech reports and
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
5.1 Intonation-unit boundaries Gene
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
(16) a. haŋ-ane // say-part Genett
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Greninger: Another look at storylin
Page 113 and 114:
# Info Type Surface Marking Grening
Page 115 and 116:
down - place -for HES HES HES house
Page 117 and 118:
Page 119 and 120:
kja¹ mi² -nok" woɾu, tɑŋ² kʰ
Page 121 and 122:
9 Background Activities Greninger:
Page 123 and 124:
and.then + sun/day after (21) SICK
Page 125 and 126:
carry -IMS soldier PL =DAT friend n
Page 127 and 128:
Page 129 and 130:
Page 131: Greninger: Another look at storylin
Page 134 and 135: Himalayan Linguistics, Vol 10(1) an
Page 136 and 137: Himalayan Linguistics, Vol 10(1) (1
Page 138 and 139: Himalayan Linguistics, Vol 10(1) Th
Page 140 and 141: sing-ATT.NMZ young.man-PL village-C
Page 144 and 145: D.DEM-DEF-ERG little.bit-little.bit
Page 148 and 149: Himalayan Linguistics, Vol 10(1) 3.
Page 152 and 153: (37) (a) *kurc-ya bɦormi stingy- S
Page 154 and 155: ‘He is always late.’ (S) (b) ho
Page 156 and 157: Himalayan Linguistics, Vol 10(1) S
Page 158 and 159: Himalayan Linguistics, Vol 10(1) No
Page 160 and 161: Himalayan Linguistics, Vol 10(1) 1
Page 164 and 165: Himalayan Linguistics, Vol 10(1) pi
Page 166 and 167: Himalayan Linguistics, Vol 10(1) In
Page 168 and 169: Himalayan Linguistics, Vol 10(1) Th
Page 170 and 171: Himalayan Linguistics, Vol 10(1) In
Page 174 and 175: Himalayan Linguistics, Vol 10(1) ch
Page 176 and 177: Himalayan Linguistics, Vol 10(1) ji
Page 180 and 181: Himalayan Linguistics, Vol 10(1) th
Page 184 and 185: Himalayan Linguistics, Vol 10(1) bo
Page 186 and 187: Himalayan Linguistics, Vol 10(1) ab
Page 188 and 189: Himalayan Linguistics, Vol 10(1) (i
Page 190 and 191: Himalayan Linguistics, Vol 10(1) th
Page 192 and 193: Himalayan Linguistics, Vol 10(1) le
Page 194 and 195: Himalayan Linguistics, Vol 10(1) Er
Page 196 and 197: Himalayan Linguistics, Vol 10(1) Re
Page 199 and 200: Himalayan Linguistics, Vol. 10(1).
Page 201 and 202: 3 Lexical correspondences with regi
Page 203 and 204: 3.2 Distinguishing features of Gyal
Page 205 and 206: Hildebrandt and Perry: Preliminary
Page 207 and 208: Chart 2. Average Initial-Syllable V
Page 209 and 210: Chart 5. Average Vowel Intensity (d
Page 217: Verbs Hildebrandt and Perry: Prelim
Page 220 and 221: Himalayan Linguistics, Vol 10(1) Fi
Page 222 and 223: Himalayan Linguistics, Vol 10(1) na
Page 224 and 225: Himalayan Linguistics, Vol 10(1) ma
Page 226 and 227: Himalayan Linguistics, Vol 10(1) (w
Page 228 and 229: Himalayan Linguistics, Vol 10(1) Mo
Page 230 and 231: Himalayan Linguistics, Vol 10(1) 48
Page 232 and 233:
Himalayan Linguistics, Vol 10(1) On
Page 234 and 235:
Himalayan Linguistics, Vol 10(1) No
Page 236 and 237:
Himalayan Linguistics, Vol 10(1) (1
Page 238 and 239:
Himalayan Linguistics, Vol 10(1) Wh
Page 240 and 241:
Himalayan Linguistics, Vol 10(1) re
Page 242 and 243:
Himalayan Linguistics, Vol 10(1) ap
Page 244 and 245:
Himalayan Linguistics, Vol 10(1) 5.
Page 246 and 247:
Himalayan Linguistics, Vol 10(1) In
Page 248 and 249:
Himalayan Linguistics, Vol 10(1) Ba
Page 250 and 251:
Page 252 and 253:
Himalayan Linguistics, Vol 10(1) ap
Page 254 and 255:
Page 256 and 257:
Himalayan Linguistics, Vol 10(1) Ap
Page 259 and 260:
Page 261 and 262:
229 Lee: Issues in Bahing orthograp
Page 263 and 264:
Page 265 and 266:
(5) [ˈb̤luʔƃa] ; [ˈjoʔƃa]
Page 267 and 268:
Page 269 and 270:
Page 271 and 272:
Page 273 and 274:
Page 275 and 276:
(28) Words with Contrastive Short a
Page 277 and 278:
Page 279 and 280:
Page 281 and 282:
Page 283 and 284:
4 Reflections 251 Lee: Issues in Ba
Page 285 and 286:
Page 287 and 288:
255 Opgenort: Tilung and its positi
Page 289 and 290:
Page 291 and 292:
PK Hayu Sunwar Bahing Jero Wambule
Page 293 and 294:
Bahing po, Wambule pa, Dumi poʔo T
Page 295 and 296:
Page 297 and 298:
Page 299 and 300:
hear Gloss lie down one PK Hayu Sun
Page 301 and 302:
Page 303:
Page 306 and 307:
Page 308 and 309:
Page 310 and 311:
Page 312 and 313:
Page 314 and 315:
Page 316 and 317:
Page 318 and 319:
Page 320 and 321:
Himalayan Linguistics, Vol 10(1) Fi
Page 322 and 323:
Himalayan Linguistics, Vol 10(1) Si
Page 324 and 325:
Himalayan Linguistics, Vol 10(1) re
Page 326 and 327:
Page 328 and 329:
Page 331 and 332:
Page 333 and 334:
301 Bergqvist: Review of Chelliah a
show all

himalayan linguistics - UCSB Linguistics - University of California ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?