A Wordnet from the Ground Up

80 Chapter 3. Discovering Semantic RelatednessThe comparison of both evaluations performed on two different development versionsof MSR RW F shows that instead of the increasing accuracy of the measure in theWBST+H test, the percentage of wordnet relation instances remains stable. We needadditional extraction mechanisms in order to increase the percentage of the targetinstances in the results and differentiate between wordnet relations – see Chapter 4.According to the planned semi-automatic expansion of the adjective and verb partsof plWordNet, the respective MSRs were extracted using the joint corpus and theMSR RW F and MSR GRW F algorithms. The procedures followed the blueprint adoptedfor the nominal MSR. We acquired two sets, 4668 adjectival lemmas and 17990 verballemmas 19 . They came from the core plWordNet (2618 and 3239, respectively), thesmall Polish-English dictionary (Piotrowski and Saloni, 1999) and the joint corpus(those occurring ≥ 1000 times).Both MSRs were tested with WBST+H tests including 2814 QA pairs for adjectivallemmas and 5484 for verbal lemmas. The QA pairs encompass 1574 different adjectivallemmas (among them 959 occur over 1000 times in the joint corpus) and 2960 differentverbal lemmas (1902 occur more than 1000 times). Some of them occur in QA pairsmore than once but with different near-synonyms.Tables 3.13 and 3.14 show the results for different MSRs on the same tests forLUs of different frequency. For WBST+H the baseline random selection is 25%. Wedivided the analysed adjectival and verbal lemmas into two groups by their frequencyin IPIC: those occurring > 1000 and the others. The results for the first group aregiven in Table 3.13. In Table 3.14 we present results obtained for all LUs.Working with the same generated co-incidence matrices for verbs and adjectives,we compared the application of RWF with three other measures: Lin’s measure (Lin,1998), CRMI (Weeds and Weir, 2005), RFF (Geffet and Dagan, 2004). From a largenumber of proposed solutions, we selected only the measures based on lexico-syntacticfeatures. Lin’s measure was included in the set because of its significant influence onthe subsequent research. CRMI has been extensively compared with several otherapproaches showing significant improvement. RFF was chosen for the idea of featureselection present in it. RFF is calculated in two phases: in the first phase featuresare evaluated and the best 100 are selected, re-weighted and used in LU similaritycalculation in the second phase. In all three approaches the similarity computation isbased in some way on Mutual Information weighting, which is also often used by othermethods. Finally, the approach of Freitag et al. (2005) is one of the few that deal withthe similarity of adjectives and verbs.In the case of RWF, we also determined experimentally the threshold k for thenumber of features selected achieving the best results with19 Besides one-word lemmas, we only considered verbs paired with the reflexive particle się.