PaLM 540B (CoT + self-consistency)

Reported on 4 benchmarks across 2 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Methodology4 results

Transfer LearningonBBH-alg
Average (%)· 2022-10-20
62.2
best: 73.9 (code-davinci-002 175B (CoT))
Scaling Instruction-Finetuned Language Models arXiv:2210.11416
Transfer LearningonBBH-nlp
Average (%)· 2022-10-20
78.2
best: 86.3 (Qwen2.5-72B)
Scaling Instruction-Finetuned Language Models arXiv:2210.11416
Multi-Task LearningonBBH-alg
Average (%)· 2022-10-20
62.2
best: 73.9 (code-davinci-002 175B (CoT))
Scaling Instruction-Finetuned Language Models arXiv:2210.11416
Multi-Task LearningonBBH-nlp
Average (%)· 2022-10-20
78.2
best: 86.3 (Qwen2.5-72B)
Scaling Instruction-Finetuned Language Models arXiv:2210.11416