Transformer+SSA

Reported on 3 benchmarks across 1 task · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical3 results

Language ModellingonWikiText-103
Test perplexity· 2023-06-02
17.6
best: 2.4 (RETRO (7.5B))
The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles arXiv:2306.01705
Language ModellingonWikiText-103
Validation perplexity· 2023-06-02
16.91
best: 13.11 (Ensemble of All)
The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles arXiv:2306.01705
Language Modellingonenwik8
Bit per Character (BPC)· 2023-06-02
1.024
best: 1.67 (LSTM (7 layers))
The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles arXiv:2306.01705