Logical Reasoning on BIG-bench (StrategyQA)
Metric: Accuracy (higher is better)
Metric: Accuracy (higher is better)
| # | Model↕ | Accuracy▼ | Augmentations | Paper | Date↕ | Code |
|---|---|---|---|---|---|---|
| 1 | PaLM-540B (few-shot, k=5) | 73.9 | No | PaLM: Scaling Language Modeling with Pathways | 2022-04-05 | Code |
| 2 | Chinchilla-70B (few-shot, k=5) | 68.3 | No | Training Compute-Optimal Large Language Models | 2022-03-29 | Code |
| 3 | PaLM-62B (few-shot, k=5) | 65.4 | No | PaLM: Scaling Language Modeling with Pathways | 2022-04-05 | Code |
| 4 | Gopher-280B (few-shot, k=5) | 61 | No | Scaling Language Models: Methods, Analysis & Ins... | 2021-12-08 | Code |