Routing Transformer

Reported on 2 benchmarks across 2 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical2 results

Image GenerationonImageNet 64x64
Bits per dim· 2020-03-12
3.43
best: 4.351 (Logsparse (6 layers))
Efficient Content-Based Sparse Attention with Routing Transformers arXiv:2003.05997
Language ModellingonWikiText-103
Test perplexity· 2020-03-12
15.8
best: 2.4 (RETRO (7.5B))
Efficient Content-Based Sparse Attention with Routing Transformers arXiv:2003.05997