Document Summarization on CNN / Daily Mail

Metric: ROUGE-L (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	ROUGE-L▼	Extra Data	Paper	Date↕	Code
1	Scrambled code + broken (alter)	45.35	No	Universal Evasion Attacks on Summarization Scoring	2022-10-25	Code
2	PEGASUS + SummaReranker	43.87	No	SummaReranker: A Multi-Task Mixture-of-Experts R...	2022-03-13	Code
3	HAT-BART	41.52	No	Hierarchical Learning for Generation with Long S...	2021-04-15	-
4	GLM-XXLarge	41.4	No	GLM: General Language Model Pretraining with Aut...	2021-03-18	Code
5	Fourier Transformer	41.34	No	Fourier Transformer: Fast Long Range Modeling by...	2023-05-24	Code
6	Hie-BART	41.05	No	-	-	-
7	BigBird-Pegasus	40.74	No	Big Bird: Transformers for Longer Sequences	2020-07-28	Code
8	T5-11B	40.69	Yes	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
9	MatchSum (RoBERTa-base)	40.55	No	Extractive Summarization as Text Matching	2020-04-19	Code
10	MatchSum (BERT-base)	40.38	No	Extractive Summarization as Text Matching	2020-04-19	Code
11	UniLM (Abstractive Summarization)	40.34	Yes	Unified Language Model Pre-training for Natural ...	2019-05-08	Code
12	BertSumExt	39.9	Yes	Text Summarization with Pretrained Encoders	2019-08-22	Code
13	BERTSUM+Transformer	39.63	Yes	Fine-tune BERT for Extractive Summarization	2019-03-25	Code
14	Selector+Pointer Generator	38.79	No	Mixture Content Selection for Diverse Sequence G...	2019-09-04	Code
15	Bottom-Up Sum	38.34	No	Bottom-Up Abstractive Summarization	2018-08-31	Code
16	NeuSUM	37.98	No	Neural Document Summarization by Jointly Learnin...	2018-07-06	Code
17	ML + RL (Paulus et al., 2017)	36.9	No	A Deep Reinforced Model for Abstractive Summariz...	2017-05-11	Code
18	TaLK Convolutions (Deep)	36.81	No	Time-aware Large Kernel Convolutions	2020-02-08	Code
19	DynamicConv	36.73	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
20	Lead-3	36.57	No	Get To The Point: Summarization with Pointer-Gen...	2017-04-14	Code
21	LightConv	36.51	No	Pay Less Attention with Lightweight and Dynamic ...	2019-01-29	Code
22	TaLK Convolutions (Standard)	36.13	No	Time-aware Large Kernel Convolutions	2020-02-08	Code
23	Synthesizer (R+V)	35.95	No	Synthesizer: Rethinking Self-Attention in Transf...	2020-05-02	Code
24	ML + Intra-Attention (Paulus et al., 2017)	35.49	No	A Deep Reinforced Model for Abstractive Summariz...	2017-05-11	Code
25	C2F + ALTERNATE	28.8	No	-	-	-
26	GPT-2	26.58	Yes	-	-	Code

#1Scrambled code + broken (alter)SOTA
45.35
ROUGE-L· 2022-10-25
Universal Evasion Attacks on Summarization Scoring Code
#2PEGASUS + SummaRerankerSOTA
43.87
ROUGE-L· 2022-03-13
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization Code
#3HAT-BARTSOTA
41.52
ROUGE-L· 2021-04-15
Hierarchical Learning for Generation with Long Source Sequences
#4GLM-XXLargeSOTA
41.4
ROUGE-L· 2021-03-18
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Code
#5Fourier Transformer
41.34
ROUGE-L· 2023-05-24
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Code
#6Hie-BART
41.05
ROUGE-L
No paper
#7BigBird-PegasusSOTA
40.74
ROUGE-L· 2020-07-28
Big Bird: Transformers for Longer Sequences Code
#8T5-11BSOTA
40.69
ROUGE-L· Extra Data· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#9MatchSum (RoBERTa-base)
40.55
ROUGE-L· 2020-04-19
Extractive Summarization as Text Matching Code
#10MatchSum (BERT-base)
40.38
ROUGE-L· 2020-04-19
Extractive Summarization as Text Matching Code
#11UniLM (Abstractive Summarization)SOTA
40.34
ROUGE-L· Extra Data· 2019-05-08
Unified Language Model Pre-training for Natural Language Understanding and Generation Code
#12BertSumExt
39.9
ROUGE-L· Extra Data· 2019-08-22
Text Summarization with Pretrained Encoders Code
#13BERTSUM+TransformerSOTA
39.63
ROUGE-L· Extra Data· 2019-03-25
Fine-tune BERT for Extractive Summarization Code
#14Selector+Pointer Generator
38.79
ROUGE-L· 2019-09-04
Mixture Content Selection for Diverse Sequence Generation Code
#15Bottom-Up SumSOTA
38.34
ROUGE-L· 2018-08-31
Bottom-Up Abstractive Summarization Code
#16NeuSUMSOTA
37.98
ROUGE-L· 2018-07-06
Neural Document Summarization by Jointly Learning to Score and Select Sentences Code
#17ML + RL (Paulus et al., 2017)SOTA
36.9
ROUGE-L· 2017-05-11
A Deep Reinforced Model for Abstractive Summarization Code
#18TaLK Convolutions (Deep)
36.81
ROUGE-L· 2020-02-08
Time-aware Large Kernel Convolutions Code
#19DynamicConv
36.73
ROUGE-L· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#20Lead-3SOTA
36.57
ROUGE-L· 2017-04-14
Get To The Point: Summarization with Pointer-Generator Networks Code
#21LightConv
36.51
ROUGE-L· 2019-01-29
Pay Less Attention with Lightweight and Dynamic Convolutions Code
#22TaLK Convolutions (Standard)
36.13
ROUGE-L· 2020-02-08
Time-aware Large Kernel Convolutions Code
#23Synthesizer (R+V)
35.95
ROUGE-L· 2020-05-02
Synthesizer: Rethinking Self-Attention in Transformer Models Code
#24ML + Intra-Attention (Paulus et al., 2017)
35.49
ROUGE-L· 2017-05-11
A Deep Reinforced Model for Abstractive Summarization Code
#25C2F + ALTERNATE
28.8
ROUGE-L
No paper
#26GPT-2
26.58
ROUGE-L· Extra Data
No paperCode