Language Modelling on C4

Metric: Perplexity (lower is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Sort:

#	Model↕	Perplexity▲	Extra Data	Paper	Date↕	Code
1	Primer	12.35	No	Primer: Searching for Efficient Transformers for...	2021-09-17	Code
2	Zeropoint LLM.int8 13B (vector-wise + decomp)	12.45	No	LLM.int8(): 8-bit Matrix Multiplication for Tran...	2022-08-15	Code
3	T5++	12.69	No	Primer: Searching for Efficient Transformers for...	2021-09-17	Code
4	Original T5	13.25	No	Primer: Searching for Efficient Transformers for...	2021-09-17	Code
5	LLM.float32 6.7B	13.3	No	LLM.int8(): 8-bit Matrix Multiplication for Tran...	2022-08-15	Code
6	LLM.float32 2.7B	14.43	No	LLM.int8(): 8-bit Matrix Multiplication for Tran...	2022-08-15	Code
7	N-Grammer 343M	14.79	No	N-Grammer: Augmenting Transformers with latent n...	2022-07-13	Code
8	N-Grammer 288M	15.01	No	N-Grammer: Augmenting Transformers with latent n...	2022-07-13	Code
9	LLM.float32 1.3B	15.91	No	LLM.int8(): 8-bit Matrix Multiplication for Tran...	2022-08-15	Code

#1PrimerSOTA
12.35
Perplexity· 2021-09-17
Primer: Searching for Efficient Transformers for Language Modeling Code
#2Zeropoint LLM.int8 13B (vector-wise + decomp)
12.45
Perplexity· 2022-08-15
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale Code
#3T5++SOTA
12.69
Perplexity· 2021-09-17
Primer: Searching for Efficient Transformers for Language Modeling Code
#4Original T5SOTA
13.25
Perplexity· 2021-09-17
Primer: Searching for Efficient Transformers for Language Modeling Code
#5LLM.float32 6.7B
13.3
Perplexity· 2022-08-15
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale Code
#6LLM.float32 2.7B
14.43
Perplexity· 2022-08-15
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale Code
#7N-Grammer 343M
14.79
Perplexity· 2022-07-13
N-Grammer: Augmenting Transformers with latent n-grams Code
#8N-Grammer 288M
15.01
Perplexity· 2022-07-13
N-Grammer: Augmenting Transformers with latent n-grams Code
#9LLM.float32 1.3B
15.91
Perplexity· 2022-08-15
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale Code