Logical Reasoning on BIG-bench

Metric: Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Hide augmentations

#	Model↕	Accuracy▼	Augmentations	Paper	Date↕	Code
1	Chinchilla-70B (few-shot, k=5)	94	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
2	Chinchilla-70B (few-shot, k=5)	93.1	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
3	Gopher-280B (few-shot, k=5)	89.5	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
4	Chinchilla-70B (few-shot, k=5)	79	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
5	Chinchilla-70B (few-shot, k=5)	67.1	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
6	Chinchilla-70B (few-shot, k=5)	60.6	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
7	Gopher-280B (few-shot, k=5)	59.7	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
8	Gopher-280B (few-shot, k=5)	59.3	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
9	Gopher-280B (few-shot, k=5)	56.4	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
10	Gopher-280B (few-shot, k=5)	53	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
11	Chinchilla-70B (few-shot, k=5)	49.9	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
12	Gopher-280B (few-shot, k=5)	37	Yes	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
13	Gopher-280B (few-shot, k=5)	34	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
14	Gopher-280B (few-shot, k=5)	33.6	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code
15	Chinchilla-70B (few-shot, k=5)	17.6	No	Training Compute-Optimal Large Language Models	2022-03-29	Code
16	Gopher-280B (few-shot, k=5)	16.7	No	Scaling Language Models: Methods, Analysis & Ins...	2021-12-08	Code