A Wordnet from the Ground Up

4.5. Hybrid Combinations 147Snow et al. (2006) evaluated manually the first n automatically added hypernymiclinks. Because n was up to 20000 in the last experiment, only randomly selectedsamples were assessed. The applied uniform size of the samples equal to 100 forn > 1000 was too small to ascribe the results of the evaluation to the whole setswith sufficient statistical confidence. Among different types of evaluation performed,the fine-grained one seems to be the most interesting from our point of view. Theevaluators were asked: “is X a Y?”, where 〈X, Y 〉 is an added link. It is not clearin (Snow et al., 2006) whether only direct hyponym/hypernyms counted as positive.For each pair of nouns 〈i, j〉, where i is unknown, the algorithm finally selects onlyone sense of j, so only the best hit is added or evaluated. According to this setting offine-grained evaluation, the achieved precision of 84% for n = 10000 is high, but maybe hard to compare with other approaches, including ours (to follow soon), because itis given only for the best hit and the basic criterion (cited above) is not precise.Alfonseca and Manandhar (2002) and Witschel (2005) use only one knowledgesource and work locally on the hypernymy tree. Each decision is based on the propertiesof the currently processed node. Widdows (2003) considers a broader context of severalpoints in the structure but also uses only one type of evidence. Caraballo (1999)combined two types of information, patterns and MSR, but the extracted structure seemsto be too far from the proper hypernymy structure. Snow et al. (2006) combine twoknowledge sources and utilise not only vertical structure but also horizontal structureof cousins (direct and indirect co-hyponyms). The assumption was that the resultsof all classifiers can be described probabilistically – not the case for lexico-syntacticpatterns. One of their classifiers is based on processing a corpus by parsing, a step notfeasible for many natural languages.How To Combine Extraction AlgorithmsIn the previous sections we have reported on several methods of extracting lexicosemanticrelations [LSR] for Polish. None of them individually has reached the accuracylevel required in a support tool for linguists. We will now investigate combinationsof the following methods:• a measure of semantic relatedness based on the Rank Weight Function, written asMSR RW F , developed for Polish nouns and presented in (Piasecki et al., 2007a)– MSR RW F extracts closely semantically related LUs with high accuracy, butthe extracted LU pairs belong to a range of LSRs, not only to the typical wordnetrelations;• post-filtering LU pairs produced by MSR RW F with a classifier presented in theSection 4.5.1 called here C H – the percentage of LSR instances increases amongthe filtered pairs;