Transformer-LS (small)

Reported on 4 benchmarks across 1 task · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical4 results

Language ModellingonText8 dev
Bit per Character (BPC)· 2021-07-05
1.03
SOTA
Long-Short Transformer: Efficient Transformers for Language and Vision arXiv:2107.02192
Language Modellingonenwik8 dev
Bit per Character (BPC)· 2021-07-05
1.01
SOTA
Long-Short Transformer: Efficient Transformers for Language and Vision arXiv:2107.02192
Language ModellingonText8
Bit per Character (BPC)· 2021-07-05
1.09
best: 1.63 (td-LSTM (Zhang et al., 2016))
Long-Short Transformer: Efficient Transformers for Language and Vision arXiv:2107.02192
Language Modellingonenwik8
Bit per Character (BPC)· 2021-07-05
0.99
best: 1.67 (LSTM (7 layers))
Long-Short Transformer: Efficient Transformers for Language and Vision arXiv:2107.02192