Flan-PaLM 540B (3-shot, fine-tuned, CoT + SC)

Reported on 4 benchmarks across 2 tasks · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Methodology4 results

Transfer LearningonBBH-nlp
Average (%)· 2022-10-20
78.4
best: 86.3 (Qwen2.5-72B)
SOTA
Scaling Instruction-Finetuned Language Models arXiv:2210.11416
Multi-Task LearningonBBH-nlp
Average (%)· 2022-10-20
78.4
best: 86.3 (Qwen2.5-72B)
SOTA
Scaling Instruction-Finetuned Language Models arXiv:2210.11416
Transfer LearningonBBH-alg
Average (%)· 2022-10-20
66.5
best: 73.9 (code-davinci-002 175B (CoT))
Scaling Instruction-Finetuned Language Models arXiv:2210.11416
Multi-Task LearningonBBH-alg
Average (%)· 2022-10-20
66.5
best: 73.9 (code-davinci-002 175B (CoT))
Scaling Instruction-Finetuned Language Models arXiv:2210.11416