A Wordnet from the Ground Up

More documents

Recommendations

Info

4.4. Benefits of Extracted Patterns 121priori by a human evaluator 10 . It is worth considering that the limited size ofplWordNet can influence precision negatively. Some LUs are either not presentyet or their synsets and all hypernymic links are incomplete. This precision iscomputed for each element on the list of instances.2. Precision based on human judgement is evaluated according to a sample randomlydrawn from the list of instances. Due to its cost, this evaluation measurewas used only for selected experiments, see below. The error level of the samplewas 3% and the confidence level was 95% (Israel, 1992).3. Recall based on plWordNet is evaluated on the set of word pairs generated fromplWordNet. This measure does not describe the recall in relation to the corpusused. In the case of many experiments, recall is also presented in the cut-offversion.The experiments were performed on exactly the same three corpora as for MSRextraction (Section 3.4.5):1. IPIC (Przepiórkowski, 2004) with ≈ 254 million tokens,2. the corpus of the electronic edition of a Polish newspaper Rzeczpospolita fromJanuary 1993 to March 2002 (Rzeczpospolita, 2008) with ≈ 113 million tokens,3. a corpus of large texts in Polish collected from the Internet, with ≈ 214 milliontokens.In contrast with the experiments with the manually constructed patterns, there wasno limit of the set of nominal LUs processed. Only the set of possible multiword LUswas predefined to accommodate the method of the recognition of multiword LUs basedon the lexicon.We tested several configurations:ESP- – Espresso without generic patterns,ESP-nm – Espresso without generic patterns, but with the extended reliability measure(4.6),ESPmorf- – Espresso without generic patterns, but with additional morphological informationencoding part of speech and values of selected grammatical categories:case (nouns), case and degree (adjectives), aspect (verbs),10 The list had resulted from the preliminary experiments and was next kept in use because of thelimited size of plWordNet.
122 Chapter 4. Extracting Relation InstancesESPfree- – extends ESPmorf- by the representation of the free order of the instanceelements,EST- — Estratto without generic patterns, exploiting specific features of Polish, e.g.the agreement on values of selected categories is represented,EST-nm — Estratto without generic patterns, exploiting specific features of Polishlanguage and the extended reliability measures (4.6),EST+nm – the same as EST-nm but using generic patterns.If not stated otherwise, the threshold for confidence is 1.0 for all ESP systems and2.6 for EST systems. The number k of top patterns was set to k = I + 2, where I isthe iteration count. There were four iterations. In those experiments whose results wepresent, the focus was only on the hyponymy/hypernymy relation. IPIC was selected asthe main corpus, on which we ran all experiments with results presented in the tables.We ran three groups of experiments on Espresso and Estratto (Kurc, 2008). Webegan with experiments designed to analyse the influence of the proposed extended reliabilitymeasure (4.6) and six pattern schemes: ESP-, ESP-mn, ESPmorf-, ESPfree-,EST-nm and EST+nm. The question was whether they improve the results, since theymay better cope with certain characteristics of Polish. In Table 4.2, the precisionbased on human judgement is presented in the column labelled “Hum. eval.”. Thelevels of precision defined in the column group labelled “Ranking” are achieved forthe top subsets of instances described in the column group named “Inst.”. They showon how large a portion of the extracted instances we can rely, and how strongly. Take,for example, the first row: ESP- extracted the top 8% of instances with the precisionabove 70% and the top 22% with the precision 60%. The higher the numbers, thehigher concentration of positive instances in the upper part of the extracted list. Theprecision measured in relation to plWordNet is presented in the column “Prec. plWN”(the number of the extracted plWordNet instances is also given). The column labelled“Rel. R” refers to the recall calculated in relation to the result of ESP-.The results of the first group of experiments, presented in Table 4.2, allow us toconclude that the modified reliability measure (4.6) performs better either in the caseof the original Espresso scheme patterns (ESP-nm is the winner) or Estratto patternswhich take into account some properties of Polish (EST-nm had the best overall result).The situation is less clear in the case of the precision based on plWordNet – thedifferences are smaller – but still ESP-nm and EST-nm produce better results thanthe other versions; plWordNet is relatively small, however, and this could bias thecalculation. The manual evaluation showed that in fact plWordNet might be used onlyfor a very rough estimation of precision. The plWordNet-based precision of ESPnmversus EST-nm is almost identical, but EST-nm is much better in relation to the
Page 1 and 2:
A Wordnetfrom the Ground Up
Page 3 and 4:
Work financed by the Polish Ministr
Page 7 and 8:
6 Prefaceheartfelt thanks go to all
Page 9:
8 Chapter 1. Motivation, Goals, Ear
Page 12 and 13:
1.1. Motivation 11[a] special form
Page 14 and 15:
1.1. Motivation 13Affect (Strappara
Page 16 and 17:
1.2. The Goals of the plWordNet Pro
Page 18 and 19:
1.2. The Goals of the plWordNet Pro
Page 20 and 21:
1.3. Early Decisions 19Merge Model:
Page 22:
1.3. Early Decisions 214. On the ot
Page 25 and 26:
24 Chapter 2. Building a Wordnet Co
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
Page 37 and 38:
Page 39 and 40:
Page 41 and 42:
Page 43 and 44:
Page 45 and 46:
Page 47 and 48:
Page 49 and 50:
48 Chapter 3. Discovering Semantic
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
Page 59 and 60:
Page 61 and 62:
Page 63 and 64:
Page 65 and 66:
Page 67 and 68:
Page 69 and 70:
Page 71 and 72: 70 Chapter 3. Discovering Semantic
Page 103 and 104: 102 Chapter 4. Extracting Relation
Page 121: 120 Chapter 4. Extracting Relation
Page 167 and 168: 166 Chapter 5. Polish WordNet Today
Page 173 and 174:
172 Chapter 5. Polish WordNet Today
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 186 and 187:
Appendix ATests for Lexico-semantic
Page 188 and 189:
187Test for adjectives (T. IX)1. p1
Page 190 and 191:
189RelatednessTest for nouns (T. XV
Page 192 and 193:
BibliographyAgarwal, Abhaya and Alo
Page 194 and 195:
Bibliography 193on Deep Lexical Acq
Page 196 and 197:
Bibliography 195Derwojedowa, Magdal
Page 198 and 199:
Bibliography 197Grefenstette, Grego
Page 200 and 201:
Bibliography 199Kurc, Roman. (2008)
Page 202 and 203:
Bibliography 201Mohammad, Saif and
Page 204 and 205:
Bibliography 203. (2006) “The pot
Page 206 and 207:
Bibliography 205and Technology 7(1-
Page 208 and 209:
List of Tables2.1 The size of the c
Page 210 and 211:
List of Figures2.1 The LU perspecti
Page 212 and 213:
List of Figures 2114.16 Completely
Page 214 and 215:
Index 213CBC, see Clustering by Com
Page 216 and 217:
Index 215169, 177, 178, 180, 182hyp
Page 218 and 219:
Index 217mutual hypernymy, 24Mutual
Page 220 and 221:
Index 219SUMO, 14Supported Vector M
Page 222:
A language without a wordnet is at
show all

A Wordnet from the Ground Up

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?