Past Decode Reg. + AWD-LSTM-MoS + dyn. eval.

Reported on 5 benchmarks across 1 task · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical5 results

Language ModellingonWikiText-2
Test perplexity· 2018-08-14
40.3
best: 8.21 (SparseGPT (175B, 50% Sparsity))
SOTA
Improved Language Modeling by Decoding the Past arXiv:1808.05908
Language ModellingonWikiText-2
Validation perplexity· 2018-08-14
42
best: 15.69 (GPT-2 (fine-tuned))
SOTA
Improved Language Modeling by Decoding the Past arXiv:1808.05908
Language ModellingonPenn Treebank (Word Level)
Test perplexity· 2018-08-14
47.3
best: 20.5 (GPT-3 (Zero-Shot))
Improved Language Modeling by Decoding the Past arXiv:1808.05908
Language ModellingonPenn Treebank (Word Level)
Validation perplexity· 2018-08-14
48
best: 36.1 (BERT-Large-CAS)
Improved Language Modeling by Decoding the Past arXiv:1808.05908
Language ModellingonPenn Treebank (Character Level)
Bit per Character (BPC)· 2018-08-14
1.169
best: 1.38 (Bipartite Flow)
Improved Language Modeling by Decoding the Past arXiv:1808.05908