Skip Cross-Head Transformer-XL

Reported on 3 benchmarks across 1 task · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical3 results

Language ModellingonWikiText-103
Test perplexity· 2023-11-14
22.91
best: 2.4 (RETRO (7.5B))
Memory-efficient Stochastic methods for Memory-based Transformers arXiv:2311.08123
Language ModellingonWikiText-103
Validation perplexity· 2023-11-14
21.87
best: 13.11 (Ensemble of All)
Memory-efficient Stochastic methods for Memory-based Transformers arXiv:2311.08123
Language Modellingonenwik8
Bit per Character (BPC)· 2023-11-14
1.033
best: 1.67 (LSTM (7 layers))
Memory-efficient Stochastic methods for Memory-based Transformers arXiv:2311.08123