Text Summarization on CNN / Daily Mail

Metric: ROUGE-L (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	ROUGE-L▼	Extra Data	Paper	Date↕	Code
1	Scrambled code + broken (alter)	45.35	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
2	Scrambled code + broken (alter)	45.35	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
3	BRIO	44.57	No	BRIO: Bringing Order to Abstractive Summarization	2022-03-31	Code
4	Pegasus	44.45	No	Calibrating Sequence likelihood Improves Conditi...	2022-09-30	-
5	PEGASUS + SummaReranker	43.87	No	SummaReranker: A Multi-Task Mixture-of-Experts R...	2022-03-13	Code
6	PEGASUS + SummaReranker	43.87	No	SummaReranker: A Multi-Task Mixture-of-Experts R...	2022-03-13	Code
7	Scrambled code + broken	43.56	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
8	BART + SimCLS	43.54	No	SimCLS: A Simple Framework for Contrastive Learn...	2021-06-03	Code
9	SEASON	43.08	No	Salience Allocation as Guidance for Abstractive ...	2022-10-22	Code
10	ERNIE-GENLARGE (large-scale text corpora)	41.6	Yes	ERNIE-GEN: An Enhanced Multi-Flow Pre-training a...	2020-01-26	Code
11	HAT-BART	41.52	No	Hierarchical Learning for Generation with Long S...	2021-04-15	-
12	PALM	41.41	No	PALM: Pre-training an Autoencoding&Autoregressiv...	2020-04-14	Code
13	GLM-XXLarge	41.4	Yes	GLM: General Language Model Pretraining with Aut...	2021-03-18	Code
14	MUPPET BART Large	41.4	No	Muppet: Massive Multi-task Representations with ...	2021-01-26	Code
15	GLM-XXLarge	41.4	No	GLM: General Language Model Pretraining with Aut...	2021-03-18	Code
16	Fourier Transformer	41.34	No	Fourier Transformer: Fast Long Range Modeling by...	2023-05-24	Code
17	Fourier Transformer	41.34	No	Fourier Transformer: Fast Long Range Modeling by...	2023-05-24	Code
18	ProphetNet	41.3	Yes	ProphetNet: Predicting Future N-gram for Sequenc...	2020-01-13	Code
19	LongT5	41.28	No	LongT5: Efficient Text-To-Text Transformer for L...	2021-12-15	Code
20	ERNIE-GENLARGE	41.26	No	ERNIE-GEN: An Enhanced Multi-Flow Pre-training a...	2020-01-26	Code
21	BART + R-Drop	41.24	No	R-Drop: Regularized Dropout for Neural Networks	2021-06-28	Code
22	CoCoNet + CoCoPretrain	41.24	Yes	-	-	Code
23	BART+R3F	41.17	No	Better Fine-Tuning by Reducing Representational ...	2020-08-06	Code
24	PEGASUS	41.11	Yes	PEGASUS: Pre-training with Extracted Gap-sentenc...	2019-12-18	Code
25	CoCoNet	41.05	No	-	-	Code
26	Hie-BART	41.05	No	-	-	-
27	BART	40.9	No	BART: Denoising Sequence-to-Sequence Pre-trainin...	2019-10-29	Code
28	HAHSum	40.75	No	-	-	-
29	BigBird-Pegasus	40.74	No	Big Bird: Transformers for Longer Sequences	2020-07-28	Code
30	T5	40.69	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
31	T5-11B	40.69	Yes	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
32	MatchSum (RoBERTa-base)	40.55	No	Extractive Summarization as Text Matching	2020-04-19	Code
33	MatchSum	40.55	No	Extractive Summarization as Text Matching	2020-04-19	Code
34	SRformer-BART	40.4	No	Segmented Recurrent Transformer: An Efficient Se...	2023-05-24	Code
35	MatchSum (BERT-base)	40.38	No	Extractive Summarization as Text Matching	2020-04-19	Code
36	UniLM	40.34	Yes	Unified Language Model Pre-training for Natural ...	2019-05-08	Code
37	UniLM (Abstractive Summarization)	40.34	Yes	Unified Language Model Pre-training for Natural ...	2019-05-08	Code
38	Scaled-MatchSum	40.287	No	-	-	-
39	NeRoBERTa	40.2	No	-	-	-
40	UniLMv2	40.14	Yes	UniLMv2: Pseudo-Masked Language Models for Unifi...	2020-02-28	Code
41	BertSumExt	39.9	Yes	Text Summarization with Pretrained Encoders	2019-08-22	Code
42	ERNIE-GENBASE	39.68	Yes	ERNIE-GEN: An Enhanced Multi-Flow Pre-training a...	2020-01-26	Code
43	BERT-ext + abs + RL + rerank	39.64	No	Summary Level Training of Sentence Rewriting for...	2019-09-19	-
44	BERTSUM+Transformer	39.63	Yes	Fine-tune BERT for Extractive Summarization	2019-03-25	Code
45	BertSumExtAbs	39.18	Yes	Text Summarization with Pretrained Encoders	2019-08-22	Code
46	BERT-ext + RL	39.11	No	Summary Level Training of Sentence Rewriting for...	2019-09-19	-
47	PNBERT	38.85	No	Searching for Effective Neural Extractive Summar...	2019-07-08	Code
48	HIBERT	38.83	No	HIBERT: Document Level Pre-training of Hierarchi...	2019-05-16	-
49	Selector & Pointer-Generator	38.79	Yes	Mixture Content Selection for Diverse Sequence G...	2019-09-04	Code
50	Two-Stage + RL	38.79	No	Pretraining-Based Natural Language Generation fo...	2019-02-25	Code
51	Selector+Pointer Generator	38.79	No	Mixture Content Selection for Diverse Sequence G...	2019-09-04	Code
52	rnn-ext + abs + RL + rerank	38.54	No	Fast Abstractive Summarization with Reinforce-Se...	2018-05-28	Code
53	EditNet	38.36	No	An Editorial Network for Enhanced Document Summa...	2019-02-27	-
54	Bottom-Up Summarization	38.34	No	Bottom-Up Abstractive Summarization	2018-08-31	Code
55	Bottom-Up Sum	38.34	No	Bottom-Up Abstractive Summarization	2018-08-31	Code
56	NeuSUM	37.98	No	Neural Document Summarization by Jointly Learnin...	2018-07-06	Code
57	NeuSUM	37.98	No	Neural Document Summarization by Jointly Learnin...	2018-07-06	Code
58	DCA	37.92	Yes	Deep Communicating Agents for Abstractive Summar...	2018-03-27	-
59	HER	37.9	No	-	-	Code
60	Mask Attention Network	37.88	No	Mask Attention Networks: Rethinking and Strength...	2021-03-25	Code
61	rnn-ext + RL	37.76	No	Fast Abstractive Summarization with Reinforce-Se...	2018-05-28	Code
62	Subformer-base	37.7	No	-	-	-
63	BanditSum	37.6	No	BanditSum: Extractive Summarization as a Context...	2018-09-25	Code
64	Latent	37.54	No	Neural Latent Extractive Document Summarization	2018-08-22	-
65	ML+RL ROUGE+Novel, with LM	37.52	No	Improving Abstraction in Text Summarization	2018-08-23	-
66	Li et al.	37.36	No	-	-	-
67	end2end w/ inconsistency loss	37.13	No	A Unified Model for Extractive and Abstractive S...	2018-05-16	Code
68	ROUGESal+Ent RL	37.1	No	Multi-Reward Reinforced Summarization with Salie...	2018-04-17	-
69	RL + pg + cbdec	37.06	No	Closed-Book Training to Improve Summarization En...	2018-09-12	-
70	ML + RL (Paulus et al., 2017)	36.9	No	A Deep Reinforced Model for Abstractive Summariz...	2017-05-11	Code
71	TaLK Convolutions (Deep)	36.81	No	Time-aware Large Kernel Convolutions	2020-02-08	Code
72	Dynamic Conv	36.73	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
73	DynamicConv	36.73	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
74	LEAD-3	36.67	No	Abstractive Text Summarization Using Sequence-to...	2016-02-19	Code
75	Transformer	36.63	No	Attention Is All You Need	2017-06-12	Code
76	REFRESH	36.6	No	Ranking Sentences for Extractive Summarization w...	2018-02-23	Code
77	Lead-3	36.57	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
78	Lead-3 baseline	36.57	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
79	Pointer + Coverage + EntailmentGen + QuestionGen	36.54	No	Soft Layer-Specific Multi-Task Summarization wit...	2018-05-28	-
80	LightConv	36.51	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
81	Li et al.	36.47	No	-	-	-
82	PTGEN + Coverage	36.38	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
83	PTGEN + Coverage	36.38	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
84	TaLK Convolutions (Standard)	36.13	No	Time-aware Large Kernel Convolutions	2020-02-08	Code
85	Synthesizer (R+V)	35.95	No	Synthesizer: Rethinking Self-Attention in Transf...	2020-05-02	Code
86	A2Summ	35.92	No	Align and Attend: Multimodal Summarization with ...	2023-03-13	Code
87	ML + Intra-Attention (Paulus et al., 2017)	35.49	No	A Deep Reinforced Model for Abstractive Summariz...	2017-05-11	Code
88	C2F + ALTERNATE	28.8	No	-	-	-
89	CriSPO 3-shot	27.4	No	CriSPO: Multi-Aspect Critique-Suggestion-guided ...	2024-10-03	Code
90	DELTA (BLSTM)	27.3	No	DELTA: A DEep learning based Language Technology...	2019-08-02	Code
91	GPT-2	26.58	Yes	-	-	Code

#1Scrambled code + broken (alter)SOTA
45.35
ROUGE-L· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#2Scrambled code + broken (alter)
45.35
ROUGE-L· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#3BRIOSOTA
44.57
ROUGE-L· 2022-03-31
BRIO: Bringing Order to Abstractive Summarization Code
#4Pegasus
44.45
ROUGE-L· 2022-09-30
Calibrating Sequence likelihood Improves Conditional Language Generation
#5PEGASUS + SummaRerankerSOTA
43.87
ROUGE-L· 2022-03-13
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization Code
#6PEGASUS + SummaReranker
43.87
ROUGE-L· 2022-03-13
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization Code
#7Scrambled code + broken
43.56
ROUGE-L· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#8BART + SimCLSSOTA
43.54
ROUGE-L· 2021-06-03
SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization Code
#9SEASON
43.08
ROUGE-L· 2022-10-22
Salience Allocation as Guidance for Abstractive Summarization Code
#10ERNIE-GENLARGE (large-scale text corpora)SOTA
41.6
ROUGE-L· Extra Data· 2020-01-26
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Code
#11HAT-BART
41.52
ROUGE-L· 2021-04-15
Hierarchical Learning for Generation with Long Source Sequences
#12PALM
41.41
ROUGE-L· 2020-04-14
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation Code
#13GLM-XXLarge
41.4
ROUGE-L· Extra Data· 2021-03-18
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Code
#14MUPPET BART Large
41.4
ROUGE-L· 2021-01-26
Muppet: Massive Multi-task Representations with Pre-Finetuning Code
#15GLM-XXLarge
41.4
ROUGE-L· 2021-03-18
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Code
#16Fourier Transformer
41.34
ROUGE-L· 2023-05-24
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Code
#17Fourier Transformer
41.34
ROUGE-L· 2023-05-24
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Code
#18ProphetNetSOTA
41.3
ROUGE-L· Extra Data· 2020-01-13
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training Code
#19LongT5
41.28
ROUGE-L· 2021-12-15
LongT5: Efficient Text-To-Text Transformer for Long Sequences Code
#20ERNIE-GENLARGE
41.26
ROUGE-L· 2020-01-26
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Code
#21BART + R-Drop
41.24
ROUGE-L· 2021-06-28
R-Drop: Regularized Dropout for Neural Networks Code
#22CoCoNet + CoCoPretrain
41.24
ROUGE-L· Extra Data
No paperCode
#23BART+R3F
41.17
ROUGE-L· 2020-08-06
Better Fine-Tuning by Reducing Representational Collapse Code
#24PEGASUSSOTA
41.11
ROUGE-L· Extra Data· 2019-12-18
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization Code
#25CoCoNet
41.05
ROUGE-L
No paperCode
#26Hie-BART
41.05
ROUGE-L
No paper
#27BARTSOTA
40.9
ROUGE-L· 2019-10-29
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension Code
#28HAHSum
40.75
ROUGE-L
No paper
#29BigBird-Pegasus
40.74
ROUGE-L· 2020-07-28
Big Bird: Transformers for Longer Sequences Code
#30T5SOTA
40.69
ROUGE-L· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#31T5-11B
40.69
ROUGE-L· Extra Data· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#32MatchSum (RoBERTa-base)
40.55
ROUGE-L· 2020-04-19
Extractive Summarization as Text Matching Code
#33MatchSum
40.55
ROUGE-L· 2020-04-19
Extractive Summarization as Text Matching Code
#34SRformer-BART
40.4
ROUGE-L· 2023-05-24
Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model Code
#35MatchSum (BERT-base)
40.38
ROUGE-L· 2020-04-19
Extractive Summarization as Text Matching Code
#36UniLMSOTA
40.34
ROUGE-L· Extra Data· 2019-05-08
Unified Language Model Pre-training for Natural Language Understanding and Generation Code
#37UniLM (Abstractive Summarization)
40.34
ROUGE-L· Extra Data· 2019-05-08
Unified Language Model Pre-training for Natural Language Understanding and Generation Code
#38Scaled-MatchSum
40.287
ROUGE-L
No paper
#39NeRoBERTa
40.2
ROUGE-L
No paper
#40UniLMv2
40.14
ROUGE-L· Extra Data· 2020-02-28
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training Code
#41BertSumExt
39.9
ROUGE-L· Extra Data· 2019-08-22
Text Summarization with Pretrained Encoders Code
#42ERNIE-GENBASE
39.68
ROUGE-L· Extra Data· 2020-01-26
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation Code
#43BERT-ext + abs + RL + rerank
39.64
ROUGE-L· 2019-09-19
Summary Level Training of Sentence Rewriting for Abstractive Summarization
#44BERTSUM+TransformerSOTA
39.63
ROUGE-L· Extra Data· 2019-03-25
Fine-tune BERT for Extractive Summarization Code
#45BertSumExtAbs
39.18
ROUGE-L· Extra Data· 2019-08-22
Text Summarization with Pretrained Encoders Code
#46BERT-ext + RL
39.11
ROUGE-L· 2019-09-19
Summary Level Training of Sentence Rewriting for Abstractive Summarization
#47PNBERT
38.85
ROUGE-L· 2019-07-08
Searching for Effective Neural Extractive Summarization: What Works and What's Next Code
#48HIBERT
38.83
ROUGE-L· 2019-05-16
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
#49Selector & Pointer-Generator
38.79
ROUGE-L· Extra Data· 2019-09-04
Mixture Content Selection for Diverse Sequence Generation Code
#50Two-Stage + RLSOTA
38.79
ROUGE-L· 2019-02-25
Pretraining-Based Natural Language Generation for Text Summarization Code
#51Selector+Pointer Generator
38.79
ROUGE-L· 2019-09-04
Mixture Content Selection for Diverse Sequence Generation Code
#52rnn-ext + abs + RL + rerankSOTA
38.54
ROUGE-L· 2018-05-28
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting Code
#53EditNet
38.36
ROUGE-L· 2019-02-27
An Editorial Network for Enhanced Document Summarization
#54Bottom-Up Summarization
38.34
ROUGE-L· 2018-08-31
Bottom-Up Abstractive Summarization Code
#55Bottom-Up Sum
38.34
ROUGE-L· 2018-08-31
Bottom-Up Abstractive Summarization Code
#56NeuSUM
37.98
ROUGE-L· 2018-07-06
Neural Document Summarization by Jointly Learning to Score and Select Sentences Code
#57NeuSUM
37.98
ROUGE-L· 2018-07-06
Neural Document Summarization by Jointly Learning to Score and Select Sentences Code
#58DCASOTA
37.92
ROUGE-L· Extra Data· 2018-03-27
Deep Communicating Agents for Abstractive Summarization
#59HER
37.9
ROUGE-L
No paperCode
#60Mask Attention Network
37.88
ROUGE-L· 2021-03-25
Mask Attention Networks: Rethinking and Strengthen Transformer Code
#61rnn-ext + RL
37.76
ROUGE-L· 2018-05-28
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting Code
#62Subformer-base
37.7
ROUGE-L
No paper
#63BanditSum
37.6
ROUGE-L· 2018-09-25
BanditSum: Extractive Summarization as a Contextual Bandit Code
#64Latent
37.54
ROUGE-L· 2018-08-22
Neural Latent Extractive Document Summarization
#65ML+RL ROUGE+Novel, with LM
37.52
ROUGE-L· 2018-08-23
Improving Abstraction in Text Summarization
#66Li et al.
37.36
ROUGE-L
No paper
#67end2end w/ inconsistency loss
37.13
ROUGE-L· 2018-05-16
A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss Code
#68ROUGESal+Ent RL
37.1
ROUGE-L· 2018-04-17
Multi-Reward Reinforced Summarization with Saliency and Entailment
#69RL + pg + cbdec
37.06
ROUGE-L· 2018-09-12
Closed-Book Training to Improve Summarization Encoder Memory
#70ML + RL (Paulus et al., 2017)SOTA
36.9
ROUGE-L· 2017-05-11
A Deep Reinforced Model for Abstractive Summarization Code
#71TaLK Convolutions (Deep)
36.81
ROUGE-L· 2020-02-08
Time-aware Large Kernel Convolutions Code
#72Dynamic Conv
36.73
ROUGE-L· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#73DynamicConv
36.73
ROUGE-L· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#74LEAD-3SOTA
36.67
ROUGE-L· 2016-02-19
Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond Code
#75Transformer
36.63
ROUGE-L· 2017-06-12
Attention Is All You Need Code
#76REFRESH
36.6
ROUGE-L· 2018-02-23
Ranking Sentences for Extractive Summarization with Reinforcement Learning Code
#77Lead-3
36.57
ROUGE-L· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#78Lead-3 baseline
36.57
ROUGE-L· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#79Pointer + Coverage + EntailmentGen + QuestionGen
36.54
ROUGE-L· 2018-05-28
Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation
#80LightConv
36.51
ROUGE-L· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#81Li et al.
36.47
ROUGE-L
No paper
#82PTGEN + Coverage
36.38
ROUGE-L· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#83PTGEN + Coverage
36.38
ROUGE-L· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#84TaLK Convolutions (Standard)
36.13
ROUGE-L· 2020-02-08
Time-aware Large Kernel Convolutions Code
#85Synthesizer (R+V)
35.95
ROUGE-L· 2020-05-02
Synthesizer: Rethinking Self-Attention in Transformer Models Code
#86A2Summ
35.92
ROUGE-L· 2023-03-13
Align and Attend: Multimodal Summarization with Dual Contrastive Losses Code
#87ML + Intra-Attention (Paulus et al., 2017)
35.49
ROUGE-L· 2017-05-11
A Deep Reinforced Model for Abstractive Summarization Code
#88C2F + ALTERNATE
28.8
ROUGE-L
No paper
#89CriSPO 3-shot
27.4
ROUGE-L· 2024-10-03
CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generation Code
#90DELTA (BLSTM)
27.3
ROUGE-L· 2019-08-02
DELTA: A DEep learning based Language Technology plAtform Code
#91GPT-2
26.58
ROUGE-L· Extra Data
No paperCode