A Wordnet from the Ground Up

72 Chapter 3. Discovering Semantic RelatednessDuring the experiments performed by (Broda et al., 2008), a linear combination ofseparate matrices, that is, a linear combination of two MSRs, gave better results thanthe joint matrix ANmod+AAdv+AA. However, as the issue of extracting MSRs on thebasis of the combination of separate matrices still requires more in depth research, wedo not present here a repeated experiment of this kind.The results of the manual evaluation of the constraints for nominal LUs, presentedin (Piasecki and Radziszewski, 2009), appear in Table 3.6. For each constraint templateand the appropriate list of lexical elements, the total number of matches in IPIC wascalculated and based on that a sample of matches was randomly drawn. Each match ofthe lexicalised morphosyntactic constraint in the sample was extracted as a triple: thesentence, the described LU and the lexical elements. The positions of both expressionsin the sentence were marked. The task of the evaluator (one of the co-authors) was toanalyse if the relation described by the constraint holds for the given pair in the givensentence. The sample sizes were chosen according to the method described in (Israel,1992), in such a way that the results of the sample evaluation can be ascribed to thewhole set with a 95% confidence level.ConstraintsAdjC NcC NmgC VsbCPrecision [%] 97.39 67.78 92.36 80.36Table 3.6:The accuracy of the lexico-morphosyntactic constraintsAs one could expect, the highest accuracy was achieved for the AdjC constraint,based strongly on agreement. The tagger caused the majority of the errors. In somecases an adjective located between two nouns of the same values of the analysedgrammatical categories was mistakenly associated with the wrong noun. The goodresult of NmgC was in large extent artificially increased by the aforementioned loosedefinition of the genitive nominal modifier assumed in NmgC and its evaluation. Forexample, we did not distinguish genitive arguments of a gerund which modifies thehead from the proper genitive modifiers of the head. Still, it is worth noting thatwe have achieved relatively good results of subject identification using a fairly simpleconstraint mechanism VsbC.As the majority constraints for verbal and adjectival LUs are symmetrical or verysimilar to those for nominal LUs, we expect similar accuracy.3.4.4 Transformation based on rank weightingIn the co-incidence matrix constructed in step 2 (Section 3.4.2, p. 65) as a result ofthe general MSR extraction process, each LU is described by a vector of features that