A Wordnet from the Ground Up

4.3. Generic Patterns Verified Statistically 115Only instances with the confidence measure above a threshold are selected for thenext iteration and used to induce and evaluate patterns. The implicit assumption isthat the confidence measure can be calculated for the majority of instances: reliablepatterns, which are less frequent but more specific, should occur at least a few timeswith the majority of correct instances on the Web. We discard all correct instanceswhich are not covered by the Web data matching reliable patterns.It is worth emphasizing that the evaluation of patterns in the next iteration is basedon instances used to induce these patterns – not on instances which they extracted.The intuition behind the measures of reliability and confidence is that patternswhich describe the given relation well often occur with many confident instances ofthis relation – and the other way around. The difference is that instances extractedby generic patterns will get high confidence if they occur in contexts matched by thespecific patterns of good reliability in the validating corpus.The measures of reliability and confidence reduce the need for manual supervisiononce Espresso has started. In a way they define the degree to which the extractedinstances express the target relation and the patterns describe contexts. The former isan advantage over manual patterns, for which collected frequencies are mostly low andaccidental, and say a little about the quality of instances.The system of measures, instances and pattern selection are universal and do notrefer to any properties of any particular relation being extracted. Espresso can thereforebe applied to a wide range of relations. It was applied to hypernymy, meronymy andantonymy, as well to more specific relations such as person-company or person-jobtitle (Pantel and Pennacchiotti, 2006).To sum up, Pantel and Pennacchiotti (2006) list Espresso’s characteristics:• high recall together with a small decrease in the precision of extracted instances,• autonomy of work (weakly supervised algorithm) – only several instances of thegiven relation must be defined at the beginning,• independence of the size of the corpus or domain used,• a wide range of relation types that can be extracted.EstrattoEstratto (Kurc, 2008, Kurc and Piasecki, 2008) is a modification of Espresso developedmainly to cope with the significant differences between English and Polish: rich morphology,flexible word-order and the much more limited size and access to the Webresources. Let us first present one language-independent adjustment.Two issues are unclear in how the reliability and confidence measures work. First,reliability is sensitive to fluctuations in PMI values. Higher values (e.g., the effect