AWD-LSTM-MoS + dynamic eval

Reported on 4 benchmarks across 1 task · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical4 results

Language ModellingonPenn Treebank (Word Level)
Test perplexity· 2017-11-10
47.69
best: 20.5 (GPT-3 (Zero-Shot))
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model arXiv:1711.03953
Language ModellingonPenn Treebank (Word Level)
Validation perplexity· 2017-11-10
48.33
best: 36.1 (BERT-Large-CAS)
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model arXiv:1711.03953
Language ModellingonWikiText-2
Test perplexity· 2017-11-10
40.68
best: 8.21 (SparseGPT (175B, 50% Sparsity))
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model arXiv:1711.03953
Language ModellingonWikiText-2
Validation perplexity· 2017-11-10
42.41
best: 15.69 (GPT-2 (fine-tuned))
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model arXiv:1711.03953