Language Modelling on One Billion Word

Metric: Validation perplexity (lower is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Sort:

#	Model↕	Validation perplexity▲	Extra Data	Paper	Date↕	Code
1	H-Transformer-1D Nr=16 (Large)	20.25	No	H-Transformer-1D: Fast One-Dimensional Hierarchi...	2021-07-25	Code
2	Adaptive Input Very Large	22.92	No	Adaptive Input Representations for Neural Langua...	2018-09-28	Code
3	Adaptive Input Large	23.83	No	Adaptive Input Representations for Neural Langua...	2018-09-28	Code
4	H-Transformer-1D Nr=16 (Base)	23.95	No	H-Transformer-1D: Fast One-Dimensional Hierarchi...	2021-07-25	Code

#1H-Transformer-1D Nr=16 (Large)SOTA
20.25
Validation perplexity· 2021-07-25
H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences Code
#2Adaptive Input Very LargeSOTA
22.92
Validation perplexity· 2018-09-28
Adaptive Input Representations for Neural Language Modeling Code
#3Adaptive Input LargeSOTA
23.83
Validation perplexity· 2018-09-28
Adaptive Input Representations for Neural Language Modeling Code
#4H-Transformer-1D Nr=16 (Base)
23.95
Validation perplexity· 2021-07-25
H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences Code