Logical Reasoning on BIG-bench

Metric: Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide augmentations

Sort:

#	Model↕	Accuracy▼	Augmentations	Paper	Date↕	Code
1	Chinchilla-70B (few-shot, k=5)	94	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
2	Chinchilla-70B (few-shot, k=5)	93.1	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
3	Gopher-280B (few-shot, k=5)	89.5	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
4	Chinchilla-70B (few-shot, k=5)	79	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
5	Chinchilla-70B (few-shot, k=5)	67.1	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
6	Chinchilla-70B (few-shot, k=5)	60.6	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
7	Gopher-280B (few-shot, k=5)	59.7	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
8	Gopher-280B (few-shot, k=5)	59.3	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
9	Gopher-280B (few-shot, k=5)	56.4	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
10	Gopher-280B (few-shot, k=5)	53	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
11	Chinchilla-70B (few-shot, k=5)	49.9	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
12	Gopher-280B (few-shot, k=5)	37	Yes	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
13	Gopher-280B (few-shot, k=5)	34	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
14	Gopher-280B (few-shot, k=5)	33.6	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
15	Chinchilla-70B (few-shot, k=5)	17.6	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
16	Gopher-280B (few-shot, k=5)	16.7	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code

#1Chinchilla-70B (few-shot, k=5)SOTA
94
Accuracy· 2022-03-29
Training Compute-Optimal Large Language Models Code
#2Chinchilla-70B (few-shot, k=5)
93.1
Accuracy· 2022-03-29
Training Compute-Optimal Large Language Models Code
#3Gopher-280B (few-shot, k=5)SOTA
89.5
Accuracy· 2021-12-08
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Code
#4Chinchilla-70B (few-shot, k=5)
79
Accuracy· 2022-03-29
Training Compute-Optimal Large Language Models Code
#5Chinchilla-70B (few-shot, k=5)
67.1
Accuracy· 2022-03-29
Training Compute-Optimal Large Language Models Code
#6Chinchilla-70B (few-shot, k=5)
60.6
Accuracy· 2022-03-29
Training Compute-Optimal Large Language Models Code
#7Gopher-280B (few-shot, k=5)
59.7
Accuracy· 2021-12-08
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Code
#8Gopher-280B (few-shot, k=5)
59.3
Accuracy· 2021-12-08
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Code
#9Gopher-280B (few-shot, k=5)
56.4
Accuracy· 2021-12-08
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Code
#10Gopher-280B (few-shot, k=5)
53
Accuracy· 2021-12-08
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Code
#11Chinchilla-70B (few-shot, k=5)
49.9
Accuracy· 2022-03-29
Training Compute-Optimal Large Language Models Code
#12Gopher-280B (few-shot, k=5)
37
Accuracy· Augmentations· 2021-12-08
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Code
#13Gopher-280B (few-shot, k=5)
34
Accuracy· 2021-12-08
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Code
#14Gopher-280B (few-shot, k=5)
33.6
Accuracy· 2021-12-08
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Code
#15Chinchilla-70B (few-shot, k=5)
17.6
Accuracy· 2022-03-29
Training Compute-Optimal Large Language Models Code
#16Gopher-280B (few-shot, k=5)
16.7
Accuracy· 2021-12-08
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Code