Language Modelling on OpenWebText

Metric: eval_perplexity (lower is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Sort:

#	Model↕	eval_perplexity▲	Extra Data	Paper	Date↕	Code
1	MDLM-Prime	15.36	No	Beyond Masked and Unmasked: Discrete Diffusion M...	2025-05-24	-
2	ARM	17.54	No	Simple and Effective Masked Diffusion Language M...	2024-06-11	Code
3	EDLM-coAR	17.58	No	Energy-Based Diffusion Language Models for Text ...	2024-10-28	-
4	GPT2-Hermite	18.39	No	Polynomial, trigonometric, and tropical activati...	2025-02-03	Code
5	GPT2-Tropical	18.64	No	Polynomial, trigonometric, and tropical activati...	2025-02-03	Code
6	GPT2-Fourier	18.72	No	Polynomial, trigonometric, and tropical activati...	2025-02-03	Code
7	GPT2-GELU	19.24	No	Polynomial, trigonometric, and tropical activati...	2025-02-03	Code
8	BD3-LMs	20.73	No	Block Diffusion: Interpolating Between Autoregre...	2025-03-12	Code
9	EDLM-NCE	21.52	No	Energy-Based Diffusion Language Models for Text ...	2024-10-28	-
10	GenMD4	21.8	No	Simplified and Generalized Masked Diffusion for ...	2024-06-06	Code
11	MDLM	22.98	No	Simple and Effective Masked Diffusion Language M...	2024-06-11	Code
12	SEDD	24.1	No	Discrete Diffusion Modeling by Estimating the Ra...	2023-10-25	Code

#1MDLM-PrimeSOTA
15.36
eval_perplexity· 2025-05-24
Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
#2ARMSOTA
17.54
eval_perplexity· 2024-06-11
Simple and Effective Masked Diffusion Language Models Code
#3EDLM-coAR
17.58
eval_perplexity· 2024-10-28
Energy-Based Diffusion Language Models for Text Generation
#4GPT2-Hermite
18.39
eval_perplexity· 2025-02-03
Polynomial, trigonometric, and tropical activations Code
#5GPT2-Tropical
18.64
eval_perplexity· 2025-02-03
Polynomial, trigonometric, and tropical activations Code
#6GPT2-Fourier
18.72
eval_perplexity· 2025-02-03
Polynomial, trigonometric, and tropical activations Code
#7GPT2-GELU
19.24
eval_perplexity· 2025-02-03
Polynomial, trigonometric, and tropical activations Code
#8BD3-LMs
20.73
eval_perplexity· 2025-03-12
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models Code
#9EDLM-NCE
21.52
eval_perplexity· 2024-10-28
Energy-Based Diffusion Language Models for Text Generation
#10GenMD4SOTA
21.8
eval_perplexity· 2024-06-06
Simplified and Generalized Masked Diffusion for Discrete Data Code
#11MDLM
22.98
eval_perplexity· 2024-06-11
Simple and Effective Masked Diffusion Language Models Code
#12SEDDSOTA
24.1
eval_perplexity· 2023-10-25
Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution Code