SRU++ Large

Reported on 4 benchmarks across 1 task · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical4 results

Language ModellingonWikiText-103
Test perplexity· 2021-02-24
17.1
best: 2.4 (RETRO (7.5B))
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute arXiv:2102.12459
Language ModellingonWikiText-103
Validation perplexity· 2021-02-24
16.4
best: 13.11 (Ensemble of All)
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute arXiv:2102.12459
Language ModellingonOne Billion Word
PPL· 2021-02-24
23.5
best: 20.09 (MDLM (AR baseline))
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute arXiv:2102.12459
Language Modellingonenwik8
Bit per Character (BPC)· 2021-02-24
0.95
best: 1.67 (LSTM (7 layers))
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute arXiv:2102.12459