Language Modelling on LAMBADA

Metric: Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

#	Model↕	Accuracy▼	Extra Data	Paper	Date↕	Code
1	PaLM-540B (Few-Shot)	89.7	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
2	PaLM 2-L (one-shot)	86.9	No	PaLM 2 Technical Report	2023-05-17	Code
3	GPT-3 175B (Few-Shot)	86.4	No	Language Models are Few-Shot Learners	2020-05-28	Code
4	LLaMA-65B+CFG (Zero-Shot)	84	No	Stay on topic with Classifier-Free Guidance	2023-06-30	-
5	LLaMA-30B+CFG (zero-shot)	83.9	No	Stay on topic with Classifier-Free Guidance	2023-06-30	-
6	PaLM 2-M (one-shot)	83.7	No	PaLM 2 Technical Report	2023-05-17	Code
7	Cohere Large	82.33	No	-	-	-
8	LLaMA-13B+CFG (zero-shot)	82.2	No	Stay on topic with Classifier-Free Guidance	2023-06-30	-
9	PaLM-540B (One-Shot)	81.8	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
10	GLaM 62B/64E (One-Shot)	80.9	No	GLaM: Efficient Scaling of Language Models with ...	2021-12-13	-
11	PaLM 2-S (one-shot)	80.7	No	PaLM 2 Technical Report	2023-05-17	Code
12	GLM-130B (bidirectional attention)	80.2	No	GLM-130B: An Open Bilingual Pre-trained Model	2022-10-05	Code
13	SparseGPT (175B, 2:4 Sparsity)	79.47	No	SparseGPT: Massive Language Models Can Be Accura...	2023-01-02	Code
14	SparseGPT (175B, 4:8 Sparsity)	78.77	No	SparseGPT: Massive Language Models Can Be Accura...	2023-01-02	Code
15	PaLM-540B (Zero-Shot)	77.9	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
16	Chinchilla (Zero-Shot)	77.7	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
17	SparseGPT (175B, 50% Sparsity)	76.51	No	SparseGPT: Massive Language Models Can Be Accura...	2023-01-02	Code
18	GPT-3 175B (Zero-Shot)	76.2	No	Language Models are Few-Shot Learners	2020-05-28	Code
19	OPT-175B	75.59	No	SparseGPT: Massive Language Models Can Be Accura...	2023-01-02	Code
20	GPT-3 13B (Zero-Shot)	72.5	No	Language Models are Few-Shot Learners	2020-05-28	Code
21	GLM-XXLarge (bidirectional)	72.35	No	GLM: General Language Model Pretraining with Aut...	2021-03-18	Code
22	Pythia 12B (0-shot)	70.46	No	Pythia: A Suite for Analyzing Large Language Mod...	2023-04-03	Code
23	GPT-3 6.7B (Zero-Shot)	70.3	No	Language Models are Few-Shot Learners	2020-05-28	Code
24	GPT-J-6B	69.7	No	-	-	-
25	Mamba-2.8B	69.2	No	Mamba: Linear-Time Sequence Modeling with Select...	2023-12-01	Code
26	Pythia 6.9B (0-shot)	67.28	No	Pythia: A Suite for Analyzing Large Language Mod...	2023-04-03	Code
27	GLM-XXLarge (unidirectional)	67.18	No	GLM: General Language Model Pretraining with Aut...	2021-03-18	Code
28	GPT-3 2.7B (Zero-Shot)	67.1	No	Language Models are Few-Shot Learners	2020-05-28	Code
29	GPT-2 1.5B (Zero Shot)	63.24	No	-	-	Code
30	Universal Transformer (w/ dynamic halting)	56.25	No	Universal Transformers	2018-07-10	Code
31	Residual Shuffle-Exchange network	54.34	No	Residual Shuffle-Exchange Networks for Fast Proc...	2020-04-06	Code
32	Gated-Attention Reader (+ features)	49	No	Broad Context Language Modeling as Reading Compr...	2016-10-26	-
33	OPT-175B (50% Sparsity)	0.02	No	SparseGPT: Massive Language Models Can Be Accura...	2023-01-02	Code
34	test	0.01	No	Test-Time Training with Self-Supervision for Gen...	2019-09-29	Code