Transformer-XL Large

Reported on 3 benchmarks across 1 task · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical3 results

Language ModellingonWikiText-103
Test perplexity· 2019-01-09
18.3
best: 2.4 (RETRO (7.5B))
SOTA
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context arXiv:1901.02860
Language ModellingonOne Billion Word
PPL· 2019-01-09
21.8
best: 20.09 (MDLM (AR baseline))
SOTA
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context arXiv:1901.02860
Language ModellingonWikiText-103
Validation perplexity· 2019-01-09
18.2
best: 13.11 (Ensemble of All)
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context arXiv:1901.02860