Cluster-Former (#C=512)

Reported on 5 benchmarks across 3 tasks · 1 paper · 4 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing4 results

Question AnsweringonQuasart-T
EM· 2020-09-13
54
SOTA
Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding arXiv:2009.06097
Question AnsweringonNatural Questions (long)
F1· 2020-09-13
76.5
best: 79.6 (DensePhrases)
SOTA
Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding arXiv:2009.06097
Question AnsweringonSearchQA
EM· 2020-09-13
68
SOTA
Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding arXiv:2009.06097
Open-Domain Question AnsweringonSearchQA
EM· 2020-09-13
68
SOTA
Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding arXiv:2009.06097

Medical1 result

Language Modellingonenwik8
Bit per Character (BPC)· 2020-09-13
1.22
best: 1.67 (LSTM (7 layers))
Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding arXiv:2009.06097