Extractive Summarization of Development Emails

need for reading them. Murray et al. [9], as a result of a user evaluation, realized that their summaries were good
according to human judges, but their system was not so reliable according to ROUGE evaluation toolkit. Drawing a
conclusion from the analyzed research, we consider extraction for the implementation of our system, and abandon
the abstractive option.
2.6 Considered techniques
We consider our approach as a mixture of all the methodological steps we found to be valuable while reading the
documents about the topic. The ideas we decide to embrace are the following:
• The employment of human testers to gather useful features to be used in the computations aimed at extracting
the sentences from a text (as in the cases [10] and [8]).
• The employment of graduate or undergraduate students in the experiment, because the probability they understand
the topics treated in the emails is higher ([6], [4]).
• Limiting the summary length written by human testers, because too long summaries do not solve the problem
of reading very long text ([4], [10], [8]).
• The requirement of dividing the selected sentences into “essential" and “optional" and give different score to a
sentence according to its category ([4]).
• Human classification of sentences to assign a binary score to every phrase, which will be useful in machine
learning phase ([10]).
• Giving the human testers pre and post questionnaires to check if their choices are influenced by some external
factor like tiredness ([7], [9]).
• Machine learning ([10], [8]), because we observed that the best working systems are the result of a procedure
involving it.
We strive for creating an approach similar to the one proposed by Rambow et al. [10]. The differences are that
they aimed at summarizing whole threads, while we aim at summarizing single messages, and that we implement
also a software system.
10