Text Summarization on CNN / Daily Mail

Metric: ROUGE-1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	ROUGE-1▼	Extra Data	Paper	Date↕	Code
1	Scrambled code + broken (alter)	48.18	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
2	Scrambled code + broken (alter)	48.18	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
3	BRIO	47.78	No	BRIO: Bringing Order to Abstractive Summarization	2022-03-31	Code
4	Pegasus	47.36	No	Calibrating Sequence likelihood Improves Conditi...	2022-09-30	-
5	PEGASUS + SummaReranker	47.16	No	SummaReranker: A Multi-Task Mixture-of-Experts R...	2022-03-13	Code
6	PEGASUS + SummaReranker	47.16	No	SummaReranker: A Multi-Task Mixture-of-Experts R...	2022-03-13	Code
7	Scrambled code + broken	46.71	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
8	BART + SimCLS	46.67	No	SimCLS: A Simple Framework for Contrastive Learn...	2021-06-03	Code
9	SEASON	46.27	No	Salience Allocation as Guidance for Abstractive ...	2022-10-22	Code
10	Fourier Transformer	44.76	No	Fourier Transformer: Fast Long Range Modeling by...	2023-05-24	Code
11	Fourier Transformer	44.76	No	Fourier Transformer: Fast Long Range Modeling by...	2023-05-24	Code
12	GLM-XXLarge	44.7	Yes	GLM: General Language Model Pretraining with Aut...	2021-03-18	Code
13	GLM-XXLarge	44.7	No	GLM: General Language Model Pretraining with Aut...	2021-03-18	Code
14	HAHSum	44.68	No	-	-	-
15	BART + R-Drop	44.51	No	R-Drop: Regularized Dropout for Neural Networks	2021-06-28	Code
16	Scaled-MatchSum	44.51	No	-	-	-
17	CoCoNet + CoCoPretrain	44.5	Yes	-	-	Code
18	HAT-BART	44.48	No	Hierarchical Learning for Generation with Long S...	2021-04-15	-
19	MUPPET BART Large	44.45	No	Muppet: Massive Multi-task Representations with ...	2021-01-26	Code
20	MatchSum (RoBERTa-base)	44.41	No	Extractive Summarization as Text Matching	2020-04-19	Code
21	MatchSum	44.41	No	Extractive Summarization as Text Matching	2020-04-19	Code
22	CoCoNet	44.39	No	-	-	Code
23	BART+R3F	44.38	No	Better Fine-Tuning by Reducing Representational ...	2020-08-06	Code
24	Hie-BART	44.35	No	-	-	-
25	ERNIE-GENLARGE (large-scale text corpora)	44.31	Yes	ERNIE-GEN: An Enhanced Multi-Flow Pre-training a...	2020-01-26	Code
26	PALM	44.3	No	PALM: Pre-training an Autoencoding&Autoregressiv...	2020-04-14	Code
27	MatchSum (BERT-base)	44.22	No	Extractive Summarization as Text Matching	2020-04-19	Code
28	ProphetNet	44.2	Yes	ProphetNet: Predicting Future N-gram for Sequenc...	2020-01-13	Code
29	PEGASUS	44.17	Yes	PEGASUS: Pre-training with Extracted Gap-sentenc...	2019-12-18	Code
30	BART	44.16	No	BART: Denoising Sequence-to-Sequence Pre-trainin...	2019-10-29	Code
31	A2Summ	44.11	No	Align and Attend: Multimodal Summarization with ...	2023-03-13	Code
32	ERNIE-GENLARGE	44.02	No	ERNIE-GEN: An Enhanced Multi-Flow Pre-training a...	2020-01-26	Code
33	LongT5	43.94	No	LongT5: Efficient Text-To-Text Transformer for L...	2021-12-15	Code
34	NeRoBERTa	43.86	No	-	-	-
35	BertSumExt	43.85	Yes	Text Summarization with Pretrained Encoders	2019-08-22	Code
36	BigBird-Pegasus	43.84	No	Big Bird: Transformers for Longer Sequences	2020-07-28	Code
37	T5	43.52	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
38	T5-11B	43.52	Yes	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
39	BERTSUM+Transformer	43.25	Yes	Fine-tune BERT for Extractive Summarization	2019-03-25	Code
40	SRformer-BART	43.19	No	Segmented Recurrent Transformer: An Efficient Se...	2023-05-24	Code
41	UniLMv2	43.16	Yes	UniLMv2: Pseudo-Masked Language Models for Unifi...	2020-02-28	Code
42	UniLM	43.08	Yes	Unified Language Model Pre-training for Natural ...	2019-05-08	Code
43	UniLM (Abstractive Summarization)	43.08	Yes	Unified Language Model Pre-training for Natural ...	2019-05-08	Code
44	BERT-ext + RL	42.76	No	Summary Level Training of Sentence Rewriting for...	2019-09-19	-
45	PNBERT	42.69	No	Searching for Effective Neural Extractive Summar...	2019-07-08	Code
46	HIBERT	42.37	No	HIBERT: Document Level Pre-training of Hierarchi...	2019-05-16	-
47	ERNIE-GENBASE	42.3	Yes	ERNIE-GEN: An Enhanced Multi-Flow Pre-training a...	2020-01-26	Code
48	HER	42.3	No	-	-	Code
49	BertSumExtAbs	42.13	Yes	Text Summarization with Pretrained Encoders	2019-08-22	Code
50	BERT-ext + abs + RL + rerank	41.9	No	Summary Level Training of Sentence Rewriting for...	2019-09-19	-
51	Selector & Pointer-Generator	41.72	Yes	Mixture Content Selection for Diverse Sequence G...	2019-09-04	Code
52	Selector+Pointer Generator	41.72	No	Mixture Content Selection for Diverse Sequence G...	2019-09-04	Code
53	Two-Stage + RL	41.71	No	Pretraining-Based Natural Language Generation fo...	2019-02-25	Code
54	DCA	41.69	Yes	Deep Communicating Agents for Abstractive Summar...	2018-03-27	-
55	NeuSUM	41.59	No	Neural Document Summarization by Jointly Learnin...	2018-07-06	Code
56	NeuSUM	41.59	No	Neural Document Summarization by Jointly Learnin...	2018-07-06	Code
57	Li et al.	41.54	No	-	-	-
58	BanditSum	41.5	No	BanditSum: Extractive Summarization as a Context...	2018-09-25	Code
59	rnn-ext + RL	41.47	No	Fast Abstractive Summarization with Reinforce-Se...	2018-05-28	Code
60	EditNet	41.42	No	An Editorial Network for Enhanced Document Summa...	2019-02-27	-
61	Bottom-Up Summarization	41.22	No	Bottom-Up Abstractive Summarization	2018-08-31	Code
62	Bottom-Up Sum	41.22	No	Bottom-Up Abstractive Summarization	2018-08-31	Code
63	Latent	41.05	No	Neural Latent Extractive Document Summarization	2018-08-22	-
64	Mask Attention Network	40.98	No	Mask Attention Networks: Rethinking and Strength...	2021-03-25	Code
65	Subformer-base	40.9	No	-	-	-
66	rnn-ext + abs + RL + rerank	40.88	No	Fast Abstractive Summarization with Reinforce-Se...	2018-05-28	Code
67	end2end w/ inconsistency loss	40.68	No	A Unified Model for Extractive and Abstractive S...	2018-05-16	Code
68	RL + pg + cbdec	40.66	No	Closed-Book Training to Improve Summarization En...	2018-09-12	-
69	TaLK Convolutions (Deep)	40.59	No	Time-aware Large Kernel Convolutions	2020-02-08	Code
70	ROUGESal+Ent RL	40.43	No	Multi-Reward Reinforced Summarization with Salie...	2018-04-17	-
71	LEAD-3	40.42	No	Abstractive Text Summarization Using Sequence-to...	2016-02-19	Code
72	Lead-3	40.34	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
73	Lead-3 baseline	40.34	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
74	Li et al.	40.3	No	-	-	-
75	ML+RL ROUGE+Novel, with LM	40.19	No	Improving Abstraction in Text Summarization	2018-08-23	-
76	TaLK Convolutions (Standard)	40.03	No	Time-aware Large Kernel Convolutions	2020-02-08	Code
77	REFRESH	40	No	Ranking Sentences for Extractive Summarization w...	2018-02-23	Code
78	ML + RL (Paulus et al., 2017)	39.87	No	A Deep Reinforced Model for Abstractive Summariz...	2017-05-11	Code
79	Dynamic Conv	39.84	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
80	DynamicConv	39.84	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
81	Pointer + Coverage + EntailmentGen + QuestionGen	39.81	No	Soft Layer-Specific Multi-Task Summarization wit...	2018-05-28	-
82	PTGEN + Coverage	39.53	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
83	PTGEN + Coverage	39.53	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
84	Pointer-Generator + Coverage	39.53	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
85	LightConv	39.52	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
86	Transformer	39.5	No	Attention Is All You Need	2017-06-12	Code
87	Synthesizer (R+V)	38.57	No	Synthesizer: Rethinking Self-Attention in Transf...	2020-05-02	Code
88	ML + Intra-Attention (Paulus et al., 2017)	38.3	No	A Deep Reinforced Model for Abstractive Summariz...	2017-05-11	Code
89	Summary Loop Unsup	37.7	No	The Summary Loop: Learning to Write Abstractive ...	2021-05-11	Code
90	C2F + ALTERNATE	31.1	No	-	-	-
91	ITS	30.8	No	Iterative Document Representation Learning Towar...	2018-09-27	Code
92	GPT-2	29.34	Yes	-	-	Code

#1Scrambled code + broken (alter)SOTA
48.18
ROUGE-1· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#2Scrambled code + broken (alter)
48.18
ROUGE-1· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#3BRIOSOTA
47.78
ROUGE-1· 2022-03-31
BRIO: Bringing Order to Abstractive Summarization Code
#4Pegasus
47.36
ROUGE-1· 2022-09-30
Calibrating Sequence likelihood Improves Conditional Language Generation
#5PEGASUS + SummaRerankerSOTA
47.16
ROUGE-1· 2022-03-13
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization Code
#6PEGASUS + SummaReranker
47.16
ROUGE-1· 2022-03-13
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization Code
#7Scrambled code + broken
46.71
ROUGE-1· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#8BART + SimCLSSOTA
46.67
ROUGE-1· 2021-06-03
SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization Code
#9SEASON
46.27
ROUGE-1· 2022-10-22
Salience Allocation as Guidance for Abstractive Summarization Code
#10Fourier Transformer
44.76
ROUGE-1· 2023-05-24
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Code
#11Fourier Transformer
44.76
ROUGE-1· 2023-05-24
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Code
#12GLM-XXLargeSOTA
44.7
ROUGE-1· Extra Data· 2021-03-18
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Code
#13GLM-XXLarge
44.7
ROUGE-1· 2021-03-18
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Code
#14HAHSum
44.68
ROUGE-1
No paper
#15BART + R-Drop
44.51
ROUGE-1· 2021-06-28
R-Drop: Regularized Dropout for Neural Networks Code
#16Scaled-MatchSum
44.51
ROUGE-1
No paper
#17CoCoNet + CoCoPretrain
44.5
ROUGE-1· Extra Data
No paperCode
#18HAT-BART
44.48
ROUGE-1· 2021-04-15
Hierarchical Learning for Generation with Long Source Sequences
#19MUPPET BART LargeSOTA
44.45
ROUGE-1· 2021-01-26
Muppet: Massive Multi-task Representations with Pre-Finetuning Code
#20MatchSum (RoBERTa-base)SOTA
44.41
ROUGE-1· 2020-04-19
Extractive Summarization as Text Matching Code
#21MatchSum
44.41
ROUGE-1· 2020-04-19
Extractive Summarization as Text Matching Code
#22CoCoNet
44.39
ROUGE-1
No paperCode
#23BART+R3F
44.38
ROUGE-1· 2020-08-06
Better Fine-Tuning by Reducing Representational Collapse Code
#24Hie-BART
44.35
ROUGE-1
No paper
#25ERNIE-GENLARGE (large-scale text corpora)SOTA
44.31
ROUGE-1· Extra Data· 2020-01-26
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Code
#26PALM
44.3
ROUGE-1· 2020-04-14
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation Code
#27MatchSum (BERT-base)
44.22
ROUGE-1· 2020-04-19
Extractive Summarization as Text Matching Code
#28ProphetNetSOTA
44.2
ROUGE-1· Extra Data· 2020-01-13
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training Code
#29PEGASUSSOTA
44.17
ROUGE-1· Extra Data· 2019-12-18
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization Code
#30BARTSOTA
44.16
ROUGE-1· 2019-10-29
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension Code
#31A2Summ
44.11
ROUGE-1· 2023-03-13
Align and Attend: Multimodal Summarization with Dual Contrastive Losses Code
#32ERNIE-GENLARGE
44.02
ROUGE-1· 2020-01-26
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Code
#33LongT5
43.94
ROUGE-1· 2021-12-15
LongT5: Efficient Text-To-Text Transformer for Long Sequences Code
#34NeRoBERTa
43.86
ROUGE-1
No paper
#35BertSumExtSOTA
43.85
ROUGE-1· Extra Data· 2019-08-22
Text Summarization with Pretrained Encoders Code
#36BigBird-Pegasus
43.84
ROUGE-1· 2020-07-28
Big Bird: Transformers for Longer Sequences Code
#37T5
43.52
ROUGE-1· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#38T5-11B
43.52
ROUGE-1· Extra Data· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#39BERTSUM+TransformerSOTA
43.25
ROUGE-1· Extra Data· 2019-03-25
Fine-tune BERT for Extractive Summarization Code
#40SRformer-BART
43.19
ROUGE-1· 2023-05-24
Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model Code
#41UniLMv2
43.16
ROUGE-1· Extra Data· 2020-02-28
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training Code
#42UniLM
43.08
ROUGE-1· Extra Data· 2019-05-08
Unified Language Model Pre-training for Natural Language Understanding and Generation Code
#43UniLM (Abstractive Summarization)
43.08
ROUGE-1· Extra Data· 2019-05-08
Unified Language Model Pre-training for Natural Language Understanding and Generation Code
#44BERT-ext + RL
42.76
ROUGE-1· 2019-09-19
Summary Level Training of Sentence Rewriting for Abstractive Summarization
#45PNBERT
42.69
ROUGE-1· 2019-07-08
Searching for Effective Neural Extractive Summarization: What Works and What's Next Code
#46HIBERT
42.37
ROUGE-1· 2019-05-16
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
#47ERNIE-GENBASE
42.3
ROUGE-1· Extra Data· 2020-01-26
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Code
#48HER
42.3
ROUGE-1
No paperCode
#49BertSumExtAbs
42.13
ROUGE-1· Extra Data· 2019-08-22
Text Summarization with Pretrained Encoders Code
#50BERT-ext + abs + RL + rerank
41.9
ROUGE-1· 2019-09-19
Summary Level Training of Sentence Rewriting for Abstractive Summarization
#51Selector & Pointer-Generator
41.72
ROUGE-1· Extra Data· 2019-09-04
Mixture Content Selection for Diverse Sequence Generation Code
#52Selector+Pointer Generator
41.72
ROUGE-1· 2019-09-04
Mixture Content Selection for Diverse Sequence Generation Code
#53Two-Stage + RLSOTA
41.71
ROUGE-1· 2019-02-25
Pretraining-Based Natural Language Generation for Text Summarization Code
#54DCASOTA
41.69
ROUGE-1· Extra Data· 2018-03-27
Deep Communicating Agents for Abstractive Summarization
#55NeuSUM
41.59
ROUGE-1· 2018-07-06
Neural Document Summarization by Jointly Learning to Score and Select Sentences Code
#56NeuSUM
41.59
ROUGE-1· 2018-07-06
Neural Document Summarization by Jointly Learning to Score and Select Sentences Code
#57Li et al.
41.54
ROUGE-1
No paper
#58BanditSum
41.5
ROUGE-1· 2018-09-25
BanditSum: Extractive Summarization as a Contextual Bandit Code
#59rnn-ext + RL
41.47
ROUGE-1· 2018-05-28
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting Code
#60EditNet
41.42
ROUGE-1· 2019-02-27
An Editorial Network for Enhanced Document Summarization
#61Bottom-Up Summarization
41.22
ROUGE-1· 2018-08-31
Bottom-Up Abstractive Summarization Code
#62Bottom-Up Sum
41.22
ROUGE-1· 2018-08-31
Bottom-Up Abstractive Summarization Code
#63Latent
41.05
ROUGE-1· 2018-08-22
Neural Latent Extractive Document Summarization
#64Mask Attention Network
40.98
ROUGE-1· 2021-03-25
Mask Attention Networks: Rethinking and Strengthen Transformer Code
#65Subformer-base
40.9
ROUGE-1
No paper
#66rnn-ext + abs + RL + rerank
40.88
ROUGE-1· 2018-05-28
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting Code
#67end2end w/ inconsistency loss
40.68
ROUGE-1· 2018-05-16
A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss Code
#68RL + pg + cbdec
40.66
ROUGE-1· 2018-09-12
Closed-Book Training to Improve Summarization Encoder Memory
#69TaLK Convolutions (Deep)
40.59
ROUGE-1· 2020-02-08
Time-aware Large Kernel Convolutions Code
#70ROUGESal+Ent RL
40.43
ROUGE-1· 2018-04-17
Multi-Reward Reinforced Summarization with Saliency and Entailment
#71LEAD-3SOTA
40.42
ROUGE-1· 2016-02-19
Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond Code
#72Lead-3
40.34
ROUGE-1· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#73Lead-3 baseline
40.34
ROUGE-1· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#74Li et al.
40.3
ROUGE-1
No paper
#75ML+RL ROUGE+Novel, with LM
40.19
ROUGE-1· 2018-08-23
Improving Abstraction in Text Summarization
#76TaLK Convolutions (Standard)
40.03
ROUGE-1· 2020-02-08
Time-aware Large Kernel Convolutions Code
#77REFRESH
40
ROUGE-1· 2018-02-23
Ranking Sentences for Extractive Summarization with Reinforcement Learning Code
#78ML + RL (Paulus et al., 2017)
39.87
ROUGE-1· 2017-05-11
A Deep Reinforced Model for Abstractive Summarization Code
#79Dynamic Conv
39.84
ROUGE-1· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#80DynamicConv
39.84
ROUGE-1· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#81Pointer + Coverage + EntailmentGen + QuestionGen
39.81
ROUGE-1· 2018-05-28
Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation
#82PTGEN + Coverage
39.53
ROUGE-1· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#83PTGEN + Coverage
39.53
ROUGE-1· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#84Pointer-Generator + Coverage
39.53
ROUGE-1· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#85LightConv
39.52
ROUGE-1· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#86Transformer
39.5
ROUGE-1· 2017-06-12
Attention Is All You Need Code
#87Synthesizer (R+V)
38.57
ROUGE-1· 2020-05-02
Synthesizer: Rethinking Self-Attention in Transformer Models Code
#88ML + Intra-Attention (Paulus et al., 2017)
38.3
ROUGE-1· 2017-05-11
A Deep Reinforced Model for Abstractive Summarization Code
#89Summary Loop Unsup
37.7
ROUGE-1· 2021-05-11
The Summary Loop: Learning to Write Abstractive Summaries Without Examples Code
#90C2F + ALTERNATE
31.1
ROUGE-1
No paper
#91ITS
30.8
ROUGE-1· 2018-09-27
Iterative Document Representation Learning Towards Summarization with Polishing Code
#92GPT-2
29.34
ROUGE-1· Extra Data
No paperCode