Text Summarization on CNN / Daily Mail

Metric: ROUGE-2 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	ROUGE-2▼	Extra Data	Paper	Date↕	Code
1	Pegasus	24.02	No	Calibrating Sequence likelihood Improves Conditi...	2022-09-30	-
2	BRIO	23.55	No	BRIO: Bringing Order to Abstractive Summarization	2022-03-31	Code
3	SEASON	22.64	No	Salience Allocation as Guidance for Abstractive ...	2022-10-22	Code
4	PEGASUS + SummaReranker	22.61	No	SummaReranker: A Multi-Task Mixture-of-Experts R...	2022-03-13	Code
5	PEGASUS + SummaReranker	22.55	No	SummaReranker: A Multi-Task Mixture-of-Experts R...	2022-03-13	Code
6	BART + SimCLS	22.15	No	SimCLS: A Simple Framework for Contrastive Learn...	2021-06-03	Code
7	BART + R-Drop	21.58	No	R-Drop: Regularized Dropout for Neural Networks	2021-06-28	Code
8	Fourier Transformer	21.55	No	Fourier Transformer: Fast Long Range Modeling by...	2023-05-24	Code
9	CoCoNet + CoCoPretrain	21.55	Yes	-	-	Code
10	T5	21.55	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
11	Fourier Transformer	21.55	No	Fourier Transformer: Fast Long Range Modeling by...	2023-05-24	Code
12	T5-11B	21.55	Yes	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
13	BART+R3F	21.53	No	Better Fine-Tuning by Reducing Representational ...	2020-08-06	Code
14	PEGASUS	21.47	Yes	PEGASUS: Pre-training with Extracted Gap-sentenc...	2019-12-18	Code
15	CoCoNet	21.41	No	-	-	Code
16	GLM-XXLarge	21.4	Yes	GLM: General Language Model Pretraining with Aut...	2021-03-18	Code
17	LongT5	21.4	No	LongT5: Efficient Text-To-Text Transformer for L...	2021-12-15	Code
18	GLM-XXLarge	21.4	No	GLM: General Language Model Pretraining with Aut...	2021-03-18	Code
19	Hie-BART	21.37	No	-	-	-
20	ERNIE-GENLARGE (large-scale text corpora)	21.35	Yes	ERNIE-GEN: An Enhanced Multi-Flow Pre-training a...	2020-01-26	Code
21	HAT-BART	21.31	No	Hierarchical Learning for Generation with Long S...	2021-04-15	-
22	HAHSum	21.3	No	-	-	-
23	BART	21.28	No	BART: Denoising Sequence-to-Sequence Pre-trainin...	2019-10-29	Code
24	MUPPET BART Large	21.25	No	Muppet: Massive Multi-task Representations with ...	2021-01-26	Code
25	ProphetNet	21.17	Yes	ProphetNet: Predicting Future N-gram for Sequenc...	2020-01-13	Code
26	ERNIE-GENLARGE	21.17	No	ERNIE-GEN: An Enhanced Multi-Flow Pre-training a...	2020-01-26	Code
27	PALM	21.12	No	PALM: Pre-training an Autoencoding&Autoregressiv...	2020-04-14	Code
28	BigBird-Pegasus	21.11	No	Big Bird: Transformers for Longer Sequences	2020-07-28	Code
29	MatchSum (RoBERTa-base)	20.86	No	Extractive Summarization as Text Matching	2020-04-19	Code
30	MatchSum	20.86	No	Extractive Summarization as Text Matching	2020-04-19	Code
31	NeRoBERTa	20.64	No	-	-	-
32	MatchSum (BERT-base)	20.62	No	Extractive Summarization as Text Matching	2020-04-19	Code
33	UniLM	20.43	Yes	Unified Language Model Pre-training for Natural ...	2019-05-08	Code
34	UniLM (Abstractive Summarization)	20.43	Yes	Unified Language Model Pre-training for Natural ...	2019-05-08	Code
35	UniLMv2	20.42	Yes	UniLMv2: Pseudo-Masked Language Models for Unifi...	2020-02-28	Code
36	Scrambled code + broken	20.39	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
37	BertSumExt	20.34	Yes	Text Summarization with Pretrained Encoders	2019-08-22	Code
38	A2Summ	20.31	No	Align and Attend: Multimodal Summarization with ...	2023-03-13	Code
39	BERTSUM+Transformer	20.24	Yes	Fine-tune BERT for Extractive Summarization	2019-03-25	Code
40	Scaled-MatchSum	20.07	No	-	-	-
41	HIBERT	19.95	No	HIBERT: Document Level Pre-training of Hierarchi...	2019-05-16	-
42	ERNIE-GENBASE	19.92	Yes	ERNIE-GEN: An Enhanced Multi-Flow Pre-training a...	2020-01-26	Code
43	BERT-ext + RL	19.87	No	Summary Level Training of Sentence Rewriting for...	2019-09-19	-
44	Scrambled code + broken (alter)	19.84	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
45	Scrambled code + broken (alter)	19.84	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
46	SRformer-BART	19.8	No	Segmented Recurrent Transformer: An Efficient Se...	2023-05-24	Code
47	BertSumExtAbs	19.6	Yes	Text Summarization with Pretrained Encoders	2019-08-22	Code
48	PNBERT	19.6	No	Searching for Effective Neural Extractive Summar...	2019-07-08	Code
49	Two-Stage + RL	19.49	No	Pretraining-Based Natural Language Generation fo...	2019-02-25	Code
50	DCA	19.47	Yes	Deep Communicating Agents for Abstractive Summar...	2018-03-27	-
51	BERT-ext + abs + RL + rerank	19.08	No	Summary Level Training of Sentence Rewriting for...	2019-09-19	-
52	EditNet	19.03	No	An Editorial Network for Enhanced Document Summa...	2019-02-27	-
53	NeuSUM	19.01	No	Neural Document Summarization by Jointly Learnin...	2018-07-06	Code
54	NeuSUM	19.01	No	Neural Document Summarization by Jointly Learnin...	2018-07-06	Code
55	TaLK Convolutions (Deep)	18.97	No	Time-aware Large Kernel Convolutions	2020-02-08	Code
56	HER	18.9	No	-	-	Code
57	Latent	18.77	No	Neural Latent Extractive Document Summarization	2018-08-22	-
58	Selector & Pointer-Generator	18.74	Yes	Mixture Content Selection for Diverse Sequence G...	2019-09-04	Code
59	Selector+Pointer Generator	18.74	No	Mixture Content Selection for Diverse Sequence G...	2019-09-04	Code
60	rnn-ext + RL	18.72	No	Fast Abstractive Summarization with Reinforce-Se...	2018-05-28	Code
61	BanditSum	18.7	No	BanditSum: Extractive Summarization as a Context...	2018-09-25	Code
62	Bottom-Up Summarization	18.68	No	Bottom-Up Abstractive Summarization	2018-08-31	Code
63	Bottom-Up Sum	18.68	No	Bottom-Up Abstractive Summarization	2018-08-31	Code
64	TaLK Convolutions (Standard)	18.45	No	Time-aware Large Kernel Convolutions	2020-02-08	Code
65	Subformer-base	18.3	No	-	-	-
66	Mask Attention Network	18.29	No	Mask Attention Networks: Rethinking and Strength...	2021-03-25	Code
67	REFRESH	18.2	No	Ranking Sentences for Extractive Summarization w...	2018-02-23	Code
68	Li et al.	18.18	No	-	-	-
69	Li et al.	18.02	No	-	-	-
70	ROUGESal+Ent RL	18	No	Multi-Reward Reinforced Summarization with Salie...	2018-04-17	-
71	end2end w/ inconsistency loss	17.97	No	A Unified Model for Extractive and Abstractive S...	2018-05-16	Code
72	RL + pg + cbdec	17.87	No	Closed-Book Training to Improve Summarization En...	2018-09-12	-
73	rnn-ext + abs + RL + rerank	17.8	No	Fast Abstractive Summarization with Reinforce-Se...	2018-05-28	Code
74	Lead-3	17.7	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
75	Lead-3 baseline	17.7	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
76	Pointer + Coverage + EntailmentGen + QuestionGen	17.64	No	Soft Layer-Specific Multi-Task Summarization wit...	2018-05-28	-
77	LEAD-3	17.62	No	Abstractive Text Summarization Using Sequence-to...	2016-02-19	Code
78	ML+RL ROUGE+Novel, with LM	17.38	No	Improving Abstraction in Text Summarization	2018-08-23	-
79	PTGEN + Coverage	17.28	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
80	PTGEN + Coverage	17.28	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
81	Pointer-Generator + Coverage	17.28	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
82	Dynamic Conv	16.25	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
83	DynamicConv	16.25	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
84	Synthesizer (R+V)	16.24	No	Synthesizer: Rethinking Self-Attention in Transf...	2020-05-02	Code
85	Transformer	16.06	No	Attention Is All You Need	2017-06-12	Code
86	LightConv	15.97	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
87	ML + RL (Paulus et al., 2017)	15.82	No	A Deep Reinforced Model for Abstractive Summariz...	2017-05-11	Code
88	C2F + ALTERNATE	15.4	No	-	-	-
89	ML + Intra-Attention (Paulus et al., 2017)	14.81	No	A Deep Reinforced Model for Abstractive Summariz...	2017-05-11	Code
90	ITS	12.6	No	Iterative Document Representation Learning Towar...	2018-09-27	Code
91	GPT-2	8.27	Yes	-	-	Code

#1PegasusSOTA
24.02
ROUGE-2· 2022-09-30
Calibrating Sequence likelihood Improves Conditional Language Generation
#2BRIOSOTA
23.55
ROUGE-2· 2022-03-31
BRIO: Bringing Order to Abstractive Summarization Code
#3SEASON
22.64
ROUGE-2· 2022-10-22
Salience Allocation as Guidance for Abstractive Summarization Code
#4PEGASUS + SummaRerankerSOTA
22.61
ROUGE-2· 2022-03-13
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization Code
#5PEGASUS + SummaReranker
22.55
ROUGE-2· 2022-03-13
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization Code
#6BART + SimCLSSOTA
22.15
ROUGE-2· 2021-06-03
SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization Code
#7BART + R-Drop
21.58
ROUGE-2· 2021-06-28
R-Drop: Regularized Dropout for Neural Networks Code
#8Fourier Transformer
21.55
ROUGE-2· 2023-05-24
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Code
#9CoCoNet + CoCoPretrain
21.55
ROUGE-2· Extra Data
No paperCode
#10T5SOTA
21.55
ROUGE-2· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#11Fourier Transformer
21.55
ROUGE-2· 2023-05-24
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Code
#12T5-11B
21.55
ROUGE-2· Extra Data· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#13BART+R3F
21.53
ROUGE-2· 2020-08-06
Better Fine-Tuning by Reducing Representational Collapse Code
#14PEGASUS
21.47
ROUGE-2· Extra Data· 2019-12-18
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization Code
#15CoCoNet
21.41
ROUGE-2
No paperCode
#16GLM-XXLarge
21.4
ROUGE-2· Extra Data· 2021-03-18
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Code
#17LongT5
21.4
ROUGE-2· 2021-12-15
LongT5: Efficient Text-To-Text Transformer for Long Sequences Code
#18GLM-XXLarge
21.4
ROUGE-2· 2021-03-18
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Code
#19Hie-BART
21.37
ROUGE-2
No paper
#20ERNIE-GENLARGE (large-scale text corpora)
21.35
ROUGE-2· Extra Data· 2020-01-26
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Code
#21HAT-BART
21.31
ROUGE-2· 2021-04-15
Hierarchical Learning for Generation with Long Source Sequences
#22HAHSum
21.3
ROUGE-2
No paper
#23BART
21.28
ROUGE-2· 2019-10-29
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension Code
#24MUPPET BART Large
21.25
ROUGE-2· 2021-01-26
Muppet: Massive Multi-task Representations with Pre-Finetuning Code
#25ProphetNet
21.17
ROUGE-2· Extra Data· 2020-01-13
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training Code
#26ERNIE-GENLARGE
21.17
ROUGE-2· 2020-01-26
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Code
#27PALM
21.12
ROUGE-2· 2020-04-14
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation Code
#28BigBird-Pegasus
21.11
ROUGE-2· 2020-07-28
Big Bird: Transformers for Longer Sequences Code
#29MatchSum (RoBERTa-base)
20.86
ROUGE-2· 2020-04-19
Extractive Summarization as Text Matching Code
#30MatchSum
20.86
ROUGE-2· 2020-04-19
Extractive Summarization as Text Matching Code
#31NeRoBERTa
20.64
ROUGE-2
No paper
#32MatchSum (BERT-base)
20.62
ROUGE-2· 2020-04-19
Extractive Summarization as Text Matching Code
#33UniLMSOTA
20.43
ROUGE-2· Extra Data· 2019-05-08
Unified Language Model Pre-training for Natural Language Understanding and Generation Code
#34UniLM (Abstractive Summarization)
20.43
ROUGE-2· Extra Data· 2019-05-08
Unified Language Model Pre-training for Natural Language Understanding and Generation Code
#35UniLMv2
20.42
ROUGE-2· Extra Data· 2020-02-28
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training Code
#36Scrambled code + broken
20.39
ROUGE-2· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#37BertSumExt
20.34
ROUGE-2· Extra Data· 2019-08-22
Text Summarization with Pretrained Encoders Code
#38A2Summ
20.31
ROUGE-2· 2023-03-13
Align and Attend: Multimodal Summarization with Dual Contrastive Losses Code
#39BERTSUM+TransformerSOTA
20.24
ROUGE-2· Extra Data· 2019-03-25
Fine-tune BERT for Extractive Summarization Code
#40Scaled-MatchSum
20.07
ROUGE-2
No paper
#41HIBERT
19.95
ROUGE-2· 2019-05-16
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
#42ERNIE-GENBASE
19.92
ROUGE-2· Extra Data· 2020-01-26
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Code
#43BERT-ext + RL
19.87
ROUGE-2· 2019-09-19
Summary Level Training of Sentence Rewriting for Abstractive Summarization
#44Scrambled code + broken (alter)
19.84
ROUGE-2· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#45Scrambled code + broken (alter)
19.84
ROUGE-2· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#46SRformer-BART
19.8
ROUGE-2· 2023-05-24
Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model Code
#47BertSumExtAbs
19.6
ROUGE-2· Extra Data· 2019-08-22
Text Summarization with Pretrained Encoders Code
#48PNBERT
19.6
ROUGE-2· 2019-07-08
Searching for Effective Neural Extractive Summarization: What Works and What's Next Code
#49Two-Stage + RLSOTA
19.49
ROUGE-2· 2019-02-25
Pretraining-Based Natural Language Generation for Text Summarization Code
#50DCASOTA
19.47
ROUGE-2· Extra Data· 2018-03-27
Deep Communicating Agents for Abstractive Summarization
#51BERT-ext + abs + RL + rerank
19.08
ROUGE-2· 2019-09-19
Summary Level Training of Sentence Rewriting for Abstractive Summarization
#52EditNet
19.03
ROUGE-2· 2019-02-27
An Editorial Network for Enhanced Document Summarization
#53NeuSUM
19.01
ROUGE-2· 2018-07-06
Neural Document Summarization by Jointly Learning to Score and Select Sentences Code
#54NeuSUM
19.01
ROUGE-2· 2018-07-06
Neural Document Summarization by Jointly Learning to Score and Select Sentences Code
#55TaLK Convolutions (Deep)
18.97
ROUGE-2· 2020-02-08
Time-aware Large Kernel Convolutions Code
#56HER
18.9
ROUGE-2
No paperCode
#57Latent
18.77
ROUGE-2· 2018-08-22
Neural Latent Extractive Document Summarization
#58Selector & Pointer-Generator
18.74
ROUGE-2· Extra Data· 2019-09-04
Mixture Content Selection for Diverse Sequence Generation Code
#59Selector+Pointer Generator
18.74
ROUGE-2· 2019-09-04
Mixture Content Selection for Diverse Sequence Generation Code
#60rnn-ext + RL
18.72
ROUGE-2· 2018-05-28
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting Code
#61BanditSum
18.7
ROUGE-2· 2018-09-25
BanditSum: Extractive Summarization as a Contextual Bandit Code
#62Bottom-Up Summarization
18.68
ROUGE-2· 2018-08-31
Bottom-Up Abstractive Summarization Code
#63Bottom-Up Sum
18.68
ROUGE-2· 2018-08-31
Bottom-Up Abstractive Summarization Code
#64TaLK Convolutions (Standard)
18.45
ROUGE-2· 2020-02-08
Time-aware Large Kernel Convolutions Code
#65Subformer-base
18.3
ROUGE-2
No paper
#66Mask Attention Network
18.29
ROUGE-2· 2021-03-25
Mask Attention Networks: Rethinking and Strengthen Transformer Code
#67REFRESHSOTA
18.2
ROUGE-2· 2018-02-23
Ranking Sentences for Extractive Summarization with Reinforcement Learning Code
#68Li et al.
18.18
ROUGE-2
No paper
#69Li et al.
18.02
ROUGE-2
No paper
#70ROUGESal+Ent RL
18
ROUGE-2· 2018-04-17
Multi-Reward Reinforced Summarization with Saliency and Entailment
#71end2end w/ inconsistency loss
17.97
ROUGE-2· 2018-05-16
A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss Code
#72RL + pg + cbdec
17.87
ROUGE-2· 2018-09-12
Closed-Book Training to Improve Summarization Encoder Memory
#73rnn-ext + abs + RL + rerank
17.8
ROUGE-2· 2018-05-28
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting Code
#74Lead-3SOTA
17.7
ROUGE-2· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#75Lead-3 baseline
17.7
ROUGE-2· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#76Pointer + Coverage + EntailmentGen + QuestionGen
17.64
ROUGE-2· 2018-05-28
Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation
#77LEAD-3SOTA
17.62
ROUGE-2· 2016-02-19
Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond Code
#78ML+RL ROUGE+Novel, with LM
17.38
ROUGE-2· 2018-08-23
Improving Abstraction in Text Summarization
#79PTGEN + Coverage
17.28
ROUGE-2· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#80PTGEN + Coverage
17.28
ROUGE-2· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#81Pointer-Generator + Coverage
17.28
ROUGE-2· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#82Dynamic Conv
16.25
ROUGE-2· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#83DynamicConv
16.25
ROUGE-2· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#84Synthesizer (R+V)
16.24
ROUGE-2· 2020-05-02
Synthesizer: Rethinking Self-Attention in Transformer Models Code
#85Transformer
16.06
ROUGE-2· 2017-06-12
Attention Is All You Need Code
#86LightConv
15.97
ROUGE-2· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#87ML + RL (Paulus et al., 2017)
15.82
ROUGE-2· 2017-05-11
A Deep Reinforced Model for Abstractive Summarization Code
#88C2F + ALTERNATE
15.4
ROUGE-2
No paper
#89ML + Intra-Attention (Paulus et al., 2017)
14.81
ROUGE-2· 2017-05-11
A Deep Reinforced Model for Abstractive Summarization Code
#90ITS
12.6
ROUGE-2· 2018-09-27
Iterative Document Representation Learning Towards Summarization with Polishing Code
#91GPT-2
8.27
ROUGE-2· Extra Data
No paperCode