A Wordnet from the Ground Up

4.3. Generic Patterns Verified Statistically 111and in the evaluation of the extracted patterns and instances (the evaluation is notpresent in some methods). Very often the process is recursive: the extracted instancesare used to generate a new pattern set.Brin (1999) proposed a method of extracting patterns that discover author-bookpairs in Web pages. Pair occurrences are represented by the order of both names andby prefix, middle and suffix character context. Patterns are generated by first groupingseed occurrences by the order and the middle. For each group, the longest matchingprefix and suffix are identified, and one pattern for a group is extracted. Patterns areevaluated by their specificity defined as the product of the lengths of the prefix, middleand suffix. Pattern with specificity below a predefined threshold are rejected 7 . Brindid not present any thorough evaluation or any accuracy data.Agichtein and Gravano (2000) as well as Agichtein et al. (2001) follow Brin’sapproach, extended with respect to the recognition of an unlimited set of relationsbetween Named Entities, and to the generation and evaluation of patterns and extractedinstances. Their system has been aptly named Snowball to reflect the iterative characterof the algorithm. It starts with the seeds and an empty set of extraction patterns. Ineach round, it extracts new patterns and new set of instances but keeps only those thathave been evaluated as sufficiently reliable. The previous set of instances becomes theseed set for the next iteration.The text is first processed by a named-entity tagger 8 . During pattern generation,text fragments including pairs of named entities in focus are extracted. For each uniquepair, the left, middle and right contexts are represented as vectors of weight in 〈0, 1〉.The weights, produced from term frequencies, are meant to express the importanceof the term for the context. Weights for the middle context are higher to reflects itslarger importance for the relation representation. The size of the left and right contextis fixed to a specified text window.During extraction, a text fragment around two named entities is transformed toa vector and compared with pattern vectors. It is accepted as an instance if the similarityof vectors (measured as a product) exceeds the threshold. Extracted patterns andinstances are evaluated by confidence. The confidence of a patterns is the ratio ofpositive to negative matches, first measured for the initial instances. Pattern confidencefor the next iteration is a combination of the new and old value. The confidence ofan instance directly depends on the confidence of the patterns that select it and on thedegree of matching between the instance and particular patterns. After each iteration,all instances with low confidence (below a threshold) are discarded.Ravichandran and Hovy (2002) developed a weakly supervised algorithm for extractionof question-answer pairs of named entities, based only on seeds. They used7 We have intentionally omitted the role of the URL addresses in pattern generation; we focus on aplain-text corpus as a source.8 The MITRE Corporation’s Alembic Workbench (Day et al., 1997) was used in Snowball.