A Wordnet from the Ground Up

84 Chapter 3. Discovering Semantic RelatednessFeatures Lin CRMI PMI RW F zscore GRW F LinNArg(acc) 60.47 58.13 55.53 56.89 63.35NArg(dat) 38.59 23.98 36.40 26.29 39.06NArg(inst) 55.60 45.59 50.75 42.54 57.57NArg(loc) 51.42 46.72 48.54 41.47 54.50Nsb 53.15 54.54 47.68 57.79 54.78VPart 46.70 44.69 44.89 47.28 48.32VAdv 65.32 53.77 58.04 64.19 66.50NArg(acc+dat+inst+loc) 65.13 65.04 60.94 64.17 68.05NSb+NArg+VPart+VAdv 67.10 67.80 62.42 62.41 71.85AAdv 57.60 20.86 53.27 58.96 58.71AA 74.24 71.86 71.75 72.32 76.87ANmod 76.12 74.77 73.99 79.18 77.75ANmod+AAdv 76.97 75.41 74.80 81.13 78.93ANmod+AA 78.18 78.32 78.43 83.05 79.89ANmod+AAdv+AA 79.71 78.32 78.39 83.26 82.48Table 3.14: Experiments with MSRs for all lemmasThe result of our best adjective MSR is very close to the result achieved by humans(Section 3.3.1). For verbs, the difference is comparable to that observed fornouns (Piasecki et al., 2007b) (but the result of verb MSR still approaches humanperformance).The constructed MSRs are intended to assist linguists in selecting LUs semanticallyrelated to the LU being edited. Lexicographers can find missing synonyms or instancesof lexico-semantic relations while browsing the MSRlist (x,k) lists (according to theMSRs).Long suggestion lists may preclude careful analysis. We chose k = 20 for a smallexperiment to test a possible future use of both MSRs by linguists. We randomlyselected two subsets of lemmas, verbs and adjectives. We determined sample sizes insuch a way that the results of the manual evaluation performed on the samples couldbe ascribed to the whole sets with the 95% confidence level, according to the methoddiscussed in (Israel, 1992). For every LU in each subset, we generated the list of thek = 20 LUs most related to the given one. One of the co-authors manually assessedall elements on all lists, distinguishing any elements that are in some wordnet relationto the head LU.The evaluated LU lists were classified into:• very useful – a half, or almost a half, of the LUs on the list are in some semanticrelation to the given one,• useful – a sizable part of the list is somehow related,