64-layer Character Transformer Model

Reported on 3 benchmarks across 1 task · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical3 results

Language ModellingonText8
Bit per Character (BPC)· 2018-08-09
1.13
best: 1.63 (td-LSTM (Zhang et al., 2016))
Character-Level Language Modeling with Deeper Self-Attention arXiv:1808.04444
Language ModellingonHutter Prize
Bit per Character (BPC)· 2018-08-09
1.06
best: 1.31 (RHN - depth 5 [zilly2016recurrent])
Character-Level Language Modeling with Deeper Self-Attention arXiv:1808.04444
Language Modellingonenwik8
Bit per Character (BPC)· 2018-08-09
1.11
best: 1.67 (LSTM (7 layers))
Character-Level Language Modeling with Deeper Self-Attention arXiv:1808.04444