The Corpus Thread - Det Danske Sprog- og Litteraturselskab

More documents

Recommendations

Info

Text metadata 20 Outline of this chapter This chapter describes how the metadata part of text items can be expressed by means of a TEI P5 header whereas Chapter 4 describes the text part proper. One major aim of the header design described in this chapter is to integrate header information from text items in existing corpora of Danish language, i.e. the Corpus of the Danish Dictionary and PAROLE-DK, KORPUS 2000, other corpus-relevant material from DOT/DSL, as well as the LGP and LSP corpora of written Danish which are compiled as part of DK-CLARIN. 3.1 Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2 Header structure . . . . . . . . . . . . . . . . . . . . . . . 22 3.2.1 The file description . . . . . . . . . . . . . . . . . 23 3.2.2 The encoding description . . . . . . . . . . . . . 33 3.2.3 The profile description . . . . . . . . . . . . . . . 36 3.2.4 The revision description . . . . . . . . . . . . . . 40 3.3 Filling in the header . . . . . . . . . . . . . . . . . . . . . 41 3.3.1 Full header template . . . . . . . . . . . . . . . . 41 3.3.2 Value sets for header standard information . . . 44 3.3.3 Additional value sets for text classification . . . . 83 Guide to reading this chapter The structure of the header is oriented towards that one used by the BNC Burnard (2007) and PAROLE-DK Keson (1998b) but tries to avoid idiosyncrasies not covered by TEI P5 as well as modifications of the TEI header schema. Section 3.1 summarizes some corpus linguistic concepts used throughout the DK-CLARIN project, which are described in further detail in Chapter 1. Section 3.2 gives a general account of the header structure of headers of text items to be included in the Corpus Text Bank, CTB. 2 The description of the CTB header structure is in its starting point strongly inspired by that one given in Burnard (2007). This section constitutes the major part of this chapter. 2 The CTB is a text repository of written texts that are candidates to be included in a linguistic corpus. The CTB has been developed by WP 2.1 in order to better process and organize potential corpus text material, see III. It must not be confused with the general DK-CLARIN repository developed in WP 5 that is supposed to support various data types (e.g. texts, images, lexicons) and various formats.
3.1. Concepts 21 Section 3.3 starts with a complete header template and describes in detail the sets of values that have to be used to fill in the header. It can be used as a manual for those who have to fill in text headers with appropriate information, either manually or automatically by converting and mapping existing material. This section is probably too detailed for those readers who just want the more general lines of how the CTB header is composed and may therefore be skipped by most readers. 3.1 Concepts A text item consists of a text potentially to be included in a corpus, and of some metadata about the text. The metadata is typically contained in a header which precedes the text proper. 3 A text item is the smallest chunk of text plus metadata in a repository of potential corpus texts – a corpus text bank – from which text items are selected for inclusion in a specific corpus. Thus, a text item is the smallest corpus-compositional unit. The text part of a text item is either a complete text (usually a shorter one) or a sample taken from a longer text, e.g. a chapter from a book, see Chapter 1. Longer texts, e.g. novels, are divided into smaller parts, e.g. chapters, before they are included in a corpus text bank. A corpus text bank may be considered as a somewhat more specialized kind of text archive, intended to contain all kinds of corpus-relevant text chunks. The reason why longer texts are chopped into smaller chunks is that this subsequently makes corpus composition more precise as text-typological fine-tuning becomes easier – a novel, for instance, is less likely to skew the intended balance of a corpus if it can be selected from the text bank in smaller quantities, e.g. chapterwise. This chapter describes the header structure of text items collected in the Corpus Text Bank (CTB) – a corpus text bank for all kinds of written corpus-relevant texts collected as part of the DK-CLARIN project’s work package 2.1: “Basic written language resources –– Reference corpus of general language”. Text items from the CTB may be included in one or more specific corpora intended for linguistic research. A corpus is a more organized collection of texts compiled on the basis of the text bank for a specific – i.e. linguistic – purpose. Text material being collected for literary purposes or as part of an electronic library (archive) may stress other features of the TEI header proposal. Here, the header structure is adopted to the specific needs of corpus texts. Text item headers are structured by means of TEI P5. In the following, this structure adapted to the needs of structurally integrating various existing corpora or text collections is described in detail. The collections to 3 Another solution would be to store the metadata in a separate database and establish a link between text and metadata.
Page 1 and 2: The Corpus Thread “Reference corp
Page 3 and 4: Chapter 1 Aim and concepts Sketchin
Page 5 and 6: 1.1. Aim of the project 5 Metadata
Page 7 and 8: 1.2. Project tasks and documentatio
Page 13 and 14: 1.3. Text collection, text bank, co
Page 15 and 16: Chapter 2 The text bank A platform
Page 17 and 18: 2.2. Implementation 17 and metadata
Page 19: Chapter 3 Text metadata What the he
Page 23 and 24: 3.2. Header structure 23 (file des
Page 25 and 26: 3.2. Header structure 25 great verb
Page 27 and 28: 3.2. Header structure 27 The follow
Page 29 and 30: 3.2. Header structure 29 deprecated
Page 31 and 32: 3.2. Header structure 31 the text i
Page 33 and 34: 3.2. Header structure 33 Another el
Page 35 and 36: 3.2. Header structure 35 appDesc
Page 37 and 38: 3.2. Header structure 37 informal p
Page 39 and 40: 3.2. Header structure 39 3.2.3.4 Te
Page 41 and 42: 3.3. Filling in the header 41 revi
Page 43 and 44: 3.3. Filling in the header 43 samp
Page 45 and 46: 3.3. Filling in the header 45 and 9
Page 47 and 48: 3.3. Filling in the header 47 descr
Page 49 and 50: 3.3. Filling in the header 49 ⊲ a
Page 51 and 52: 3.3. Filling in the header 51 Legal
Page 53 and 54: 3.3. Filling in the header 53 ⊲ a
Page 61 and 62: 3.3. Filling in the header 61 Value
Page 65 and 66: 3.3. Filling in the header 65 ⊲ p
Page 67 and 68: 3.3. Filling in the header 67 Prope
Page 71 and 72:
3.3. Filling in the header 71 Value
Page 73 and 74:
3.3. Filling in the header 73 Legal
Page 75 and 76:
Page 77 and 78:
Page 79 and 80:
Page 81 and 82:
3.3. Filling in the header 81 Value
Page 83 and 84:
3.3. Filling in the header 83 3.3.3
Page 85 and 86:
4.1. Basic considerations 85 Outlin
Page 87 and 88:
4.2. Formatting text 87 russiske hi
Page 89 and 90:
4.2. Formatting text 89 Mirganjan
Page 91 and 92:
4.2. Formatting text 91 udvikle ude
Page 93 and 94:
4.2. Formatting text 93 4.2.3.4 Add
Page 95 and 96:
4.2. Formatting text 95 i dag . T
Page 97 and 98:
Part III Collecting 97
Page 99 and 100:
5.1. Implementation 99 5.5.2 Implem
Page 101 and 102:
5.1. Implementation 101 [request c
Page 103 and 104:
5.2. Header constructor:make-header
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
5.3. Pre-tokenizer:pretokenize 115
Page 117 and 118:
5.3. Pre-tokenizer:pretokenize 117
Page 119 and 120:
5.4. Text id registry:register-text
Page 121 and 122:
5.4. Text id registry:register-text
Page 123 and 124:
5.5. id dispatcher:make-id 123 0 11
Page 125 and 126:
Part IV Markup 125
Page 127 and 128:
6.1. Requirements 127 Outline of th
Page 129 and 130:
6.2. Survey 129 1. IMS’s TreeTagg
Page 131 and 132:
6.2. Survey 131 Code: Java. Assessm
Page 133 and 134:
6.2. Survey 133 9. alias-i’s Ling
Page 135 and 136:
6.3. Case study 135 lemmatization c
Page 137 and 138:
6.4. Further reading 137 disambigua
Page 139 and 140:
7.1. Modifications of the PAROLE Co
Page 141 and 142:
7.2. The ePOS tag set for Danish 14
Page 143 and 144:
Page 145 and 146:
Page 147 and 148:
Page 149 and 150:
Page 151 and 152:
Page 153 and 154:
Chapter 8 The full-form lexicon Sam
Page 155 and 156:
8.1. Enhancing existing material 15
Page 157 and 158:
8.1. Enhancing existing material 15
Page 159 and 160:
Part V Deployment 159
Page 161 and 162:
9.2. Text material 161 Type Source
Page 163 and 164:
10.2. Funktionaliteter 163 2. The u
Page 165 and 166:
10.2. Funktionaliteter 165 fortolke
Page 167 and 168:
10.2. Funktionaliteter 167 teknolog
Page 169 and 170:
10.2. Funktionaliteter 169 Implemen
Page 171 and 172:
10.2. Funktionaliteter 171 forbinde
Page 173 and 174:
10.2. Funktionaliteter 173 10.2.9 K
Page 175 and 176:
Bibliography Andersen, M. S., Asmus
show all

The Corpus Thread - Det Danske Sprog- og Litteraturselskab

Create successful ePaper yourself

Delete template?

Save as template?