Induction of Word and Phrase Alignments for Automatic Document ...

More documents

Recommendations

Info

Computational Linguistics Volume 1, Number 1 Connecting Point has become the single largest Mac retailer after tripling it ’s Macintosh sales since January 1989 . Connecting Point Systems tripled it ’s sales of Apple Macintosh systems since last January . It is now the single largest seller of Macintosh . Figure 1 Example alignment of a single abstract sentence with two document sentences. tively simple alignment between a document fragment and its corresponding abstract fragment from our corpus. In this example, a single abstract sentence (shown along the top of the figure) corresponds to exactly two document sentences (shown along the bottom of the figure). If we are able to automatically generate such alignments, one can envision the development of models of summarization that take into account effects of word choice, phrasal and sentence reordering, and content selection. Such models could be simultaneously linguistically motivated and data-driven. Furthermore, such alignments are potentially useful for current-day summarization techniques, including sentence extraction, headline generation and document compression. A close examination of the alignment shown in Figure 1 leads us to three observations about the nature of the relationship between a document and its abstract, and hence about the alignment itself: • Alignments can occur at the granularity of words and of phrases. • The ordering of phrases in an abstract can be different from the ordering of phrases in the document. • Some abstract words do not have direct correspondents in the document, and many document words are never used in an abstract. In order to develop an alignment model that could potentially recreate such an alignment, we need our model to be able to operate both at the word level and at the phrase level, we need it to be able to allow arbitrary reorderings, and we need it to be able to account for words on both the document and abstract side that have no direct correspondence. In this paper, we develop an alignment model that is capable of learning all these aspects of the alignment problem in a completely unsupervised fashion. 1.2 Shortcomings of current summarization models Current state of the art automatic single-document summarization systems employ one of three techniques: sentence extraction, bag-of-words headline generation, or document compression. Sentence extraction systems take full sentences from a document and concatenate them to form a summary. Research in sentence extraction can be traced back to work in the mid fifties and late sixties by Luhn (1956) and Edmundson (1969). Recent techniques are startlingly not terribly divergent from these original methods; see (Mani and Maybury, 1999; Marcu, 2000; Mani, 2001) for a comprehensive overview. Headline generation systems, on the other hand, typically extract individual words from a document to produce a very short headline-style summary; see (Banko, Mittal, and Witbrock, 2000; Berger and Mittal, 2000; Schwartz, Zajic, and Dorr, 2002) for representative examples. Between these two extremes, there has been a relatively modest amount of work in sentence simplification (Chandrasekar, Doran, and Bangalore, 1996; Mahesh, 1997; Carroll et al., 1998; Grefenstette, 1998; Jing, 2000; Knight and Marcu, 2002) and document compression (Daumé III and Marcu, 2002; Daumé III and Marcu, 2004; Zajic, Dorr, and Schwartz, 2004) in which words, phrases and sentences are selected in an extraction process. 2
Alignments for Document Summarization Daumé III & Marcu While such approaches have enjoyed some success, they all suffer from modeling shortcomings. Sentence extraction systems and document compression models make unrealistic assumptions about the summarization task (namely, that extraction is sufficient, and that sentences are the appropriate level of granularity). Headline generation systems employ very weak models that make an incorrect bag-of-words assumption. This assumption allows such systems to learn limited transformations to produce headlines from arbitrary documents, but such transformations are not nearly complex enough to adequately model anything beyond indicative summaries at a length of around ten words. Bag of words models can learn what the most important words to keep in a headline are, but say nothing about how to structure them in a well-formed, grammatical headline. In our own work on document compression models (Daumé III and Marcu, 2002; Daumé III and Marcu, 2004), both of which extend the sentence compression model of Knight and Marcu (2002), we assume that sentences and documents can be summarized exclusively through deletion of contiguous text segments. In Knight and Marcu’s data, we found that from a corpus of 39, 060 abstract sentences, only 1067 were created from corresponding document sentences via deletion of contiguous segments. In other words, only 2.7% of the sentences in real 〈document, abstract〉 pairs can be explained by the model proposed by Knight and Marcu (2002). Such document compression models do not explain the rich set of linguistic devices employed, for example, in Figure 1. 1.3 Prior work on alignments In the sentence extraction community, there exist a wide variety of techniques for (essentially) creating alignments between document sentences and abstract sentences (Kupiec, Pedersen, and Chen, 1995; Teufel and Moens, 1997; Marcu, 1999); see also (Barzilay and Elhadad, 2003; Quirk, Brockett, and Dolan, 2004) for work describing alignments for the monolingual paraphrasing task. These techniques typically take into account information such as lexical overlap, synonymy, ordering, length, discourse structure, and so forth. The sentence alignment problem is a comparatively simple problem to solve and current approaches work quite well. Unfortunately, these alignments are the least useful, as they can only be used to train sentence extraction systems. In the context of headline generation, simple statistical models are used for aligning documents and headlines (Banko, Mittal, and Witbrock, 2000; Berger and Mittal, 2000; Schwartz, Zajic, and Dorr, 2002), based on IBM Model 1 (Brown et al., 1993). These models treat documents and headlines as simple bags of words and learn probabilistic word-based mappings between the words in the documents and the words in the headlines. Such mappings can be considered word-to-word alignments, but as our results show (see Section 5), these models are too weak for capturing the sophisticated operations that are employed by humans in summarizing texts. To date, there has been very little work on the word alignment task in the context of summarization. The most relevant work is that of Jing (2002), in which a hidden Markov alignment model is applied to the task of identifying word and phrase-level correspondences between documents and abstracts. Unfortunately, this model is only able to align words that are identical up to their stems, and thus suffers from a problem of recall. This also makes it ill-suited to the task of learning how to perform abstraction, in which one would desire to know how words get changed. For example, Jing’s model cannot identify any of the following alignments from Figure 1: (Connecting Point ↔ Connecting Point Systems), (Mac ↔ Macintosh), (retailer ↔ seller), (Macintosh ↔ Apple Macintosh systems) and (January 1989 ↔ last January). Word alignment (and, to a lesser degree, phrase alignment) has been an active topic of research in the machine translation community. Based on these efforts, one might 3
Page 1: Induction of Word and Phrase Alignm
Page 5 and 6: Alignments for Document Summarizati
Page 25: Alignments for Document Summarizati

Induction of Word and Phrase Alignments for Automatic Document ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?