Language Modelling on Penn Treebank (Character Level)

Metric: Bit per Character (BPC) (higher is better)

LeaderboardDataset

Loading chart...

Results

#	Model↕	Bit per Character (BPC)▼	Extra Data	Paper	Date↕	Code
1	Bipartite Flow	1.38	No	Discrete Flows: Invertible Generative Models of ...	2019-05-24	Code
2	TCN	1.31	No	Seq-U-Net: A One-Dimensional Causal U-Net for Ef...	2019-11-14	Code
3	Temporal Convolutional Network	1.31	No	An Empirical Evaluation of Generic Convolutional...	2018-03-04	Code
4	Seq-U-Net	1.3	No	Seq-U-Net: A One-Dimensional Causal U-Net for Ef...	2019-11-14	Code
5	STAR	1.3	No	Gating Revisited: Deep Multi-layer RNNs That Can...	2019-11-25	Code
6	R-Transformer	1.24	No	R-Transformer: Recurrent Neural Network Enhanced...	2019-07-12	Code
7	2-layer Norm HyperLSTM	1.219	No	HyperNetworks	2016-09-27	Code
8	NAS-RL	1.214	No	Neural Architecture Search with Reinforcement Le...	2016-11-05	Code
9	FS-LSTM-2	1.193	No	Fast-Slow Recurrent Neural Networks	2017-05-24	Code
10	FS-LSTM-4	1.19	No	Fast-Slow Recurrent Neural Networks	2017-05-24	Code
11	IndRNN	1.19	No	Independently Recurrent Neural Network (IndRNN):...	2018-03-13	Code
12	6-layer QRNN	1.187	No	An Analysis of Neural Language Modeling at Multi...	2018-03-22	Code
13	Dense IndRNN	1.18	No	Deep Independently Recurrent Neural Network (Ind...	2019-10-11	Code
14	3-layer AWD-LSTM	1.175	No	An Analysis of Neural Language Modeling at Multi...	2018-03-22	Code
15	Past Decode Reg. + AWD-LSTM-MoS + dyn. eval.	1.169	No	Improved Language Modeling by Decoding the Past	2018-08-14	-
16	Feedback Transformer	1.16	No	Addressing Some Limitations of Transformers with...	2020-02-21	Code
17	Trellis Network	1.158	No	Trellis Networks for Sequence Modeling	2018-10-15	Code
18	GAM-RHN-5	1.147	No	-	-	Code
19	Mogrifier LSTM	1.12	No	Mogrifier LSTM	2019-09-04	Code
20	Mogrifier LSTM + dynamic eval	1.083	No	Mogrifier LSTM	2019-09-04	Code