Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/Synthesizer (R+V)

Synthesizer (R+V)

Reported on 24 benchmarks across 8 tasks · 1 paper · 17 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing9 results

Semantic Textual SimilarityonMRPC Dev
Accuracy· 2020-05-02
91.2
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Dialogue GenerationonPersona-Chat
BLEU-1· uses extra data· 2020-05-02
14.7
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Dialogue GenerationonPersona-Chat
CIDr· uses extra data· 2020-05-02
19.09
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Dialogue GenerationonPersona-Chat
METEOR· uses extra data· 2020-05-02
6.39
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Dialogue GenerationonPersona-Chat
ROUGE-L· uses extra data· 2020-05-02
14.79
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Linguistic AcceptabilityonCoLA Dev
Accuracy· 2020-05-02
53.3
best: 88.6 (En-BERT + TDA)
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Document SummarizationonCNN / Daily Mail
ROUGE-1· 2020-05-02
38.57
best: 48.18 (Scrambled code + broken (alter))
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Document SummarizationonCNN / Daily Mail
ROUGE-2· 2020-05-02
16.24
best: 22.55 (PEGASUS + SummaReranker)
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Document SummarizationonCNN / Daily Mail
ROUGE-L· 2020-05-02
35.95
best: 45.35 (Scrambled code + broken (alter))
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743

Speech4 results

DialogueonPersona-Chat
BLEU-1· uses extra data· 2020-05-02
14.7
best: 49.581 (BART (TextBox 2.0))
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
DialogueonPersona-Chat
CIDr· uses extra data· 2020-05-02
19.09
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
DialogueonPersona-Chat
METEOR· uses extra data· 2020-05-02
6.39
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
DialogueonPersona-Chat
ROUGE-L· uses extra data· 2020-05-02
14.79
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743

Adversarial4 results

Text GenerationonPersona-Chat
BLEU-1· uses extra data· 2020-05-02
14.7
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Text GenerationonPersona-Chat
CIDr· uses extra data· 2020-05-02
19.09
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Text GenerationonPersona-Chat
METEOR· uses extra data· 2020-05-02
6.39
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Text GenerationonPersona-Chat
ROUGE-L· uses extra data· 2020-05-02
14.79
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743

Methodology4 results

ChatbotonPersona-Chat
BLEU-1· uses extra data· 2020-05-02
14.7
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
ChatbotonPersona-Chat
CIDr· uses extra data· 2020-05-02
19.09
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
ChatbotonPersona-Chat
METEOR· uses extra data· 2020-05-02
6.39
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
ChatbotonPersona-Chat
ROUGE-L· uses extra data· 2020-05-02
14.79
SOTA
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743

Knowledge Base3 results

Text SummarizationonCNN / Daily Mail
ROUGE-1· 2020-05-02
38.57
best: 48.18 (Scrambled code + broken (alter))
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Text SummarizationonCNN / Daily Mail
ROUGE-2· 2020-05-02
16.24
best: 24.02 (Pegasus)
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743
Text SummarizationonCNN / Daily Mail
ROUGE-L· 2020-05-02
35.95
best: 45.35 (Scrambled code + broken (alter))
Synthesizer: Rethinking Self-Attention in Transformer Models arXiv:2005.00743