Ramakanth Pasunuru, Mohit Bansal
Abstractive text summarization is the task of compressing and rewriting a long document into a short summary while maintaining saliency, directed logical entailment, and non-redundancy. In this work, we address these three important aspects of a good summary via a reinforcement learning approach with two novel reward functions: ROUGESal and Entail, on top of a coverage-based baseline. The ROUGESal reward modifies the ROUGE metric by up-weighting the salient phrases/words detected via a keyphrase classifier. The Entail reward gives high (length-normalized) scores to logically-entailed summaries using an entailment classifier. Further, we show superior performance improvement when these rewards are combined with traditional metric (ROUGE) based rewards, via our novel and effective multi-reward approach of optimizing multiple rewards simultaneously in alternate mini-batches. Our method achieves the new state-of-the-art results (including human evaluation) on the CNN/Daily Mail dataset as well as strong improvements in a test-only transfer setup on DUC-2002.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Text Summarization | CNN / Daily Mail | ROUGE-1 | 40.43 | ROUGESal+Ent RL |
| Text Summarization | CNN / Daily Mail | ROUGE-2 | 18 | ROUGESal+Ent RL |
| Text Summarization | CNN / Daily Mail | ROUGE-L | 37.1 | ROUGESal+Ent RL |
| Abstractive Text Summarization | CNN / Daily Mail | ROUGE-1 | 40.43 | ROUGESal+Ent RL |
| Abstractive Text Summarization | CNN / Daily Mail | ROUGE-2 | 18 | ROUGESal+Ent RL |
| Abstractive Text Summarization | CNN / Daily Mail | ROUGE-L | 37.1 | ROUGESal+Ent RL |