Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/PaLM 540B

PaLM 540B

Reported on 16 benchmarks across 7 tasks · 4 papers · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Methodology6 results

Transfer LearningonMGSM
Average (%)· 2022-04-05
55
best: 87 (PaLM 2 (few-shot, k=8, SC))
SOTA
PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311
Multi-Task LearningonMGSM
Average (%)· 2022-04-05
55
best: 87 (PaLM 2 (few-shot, k=8, SC))
SOTA
PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311
Transfer LearningonBBH-alg
Average (%)· 2022-10-20
38.3
best: 73.9 (code-davinci-002 175B (CoT))
Scaling Instruction-Finetuned Language Models arXiv:2210.11416
Transfer LearningonBBH-nlp
Average (%)· 2022-10-20
62.7
best: 86.3 (Qwen2.5-72B)
Scaling Instruction-Finetuned Language Models arXiv:2210.11416
Multi-Task LearningonBBH-alg
Average (%)· 2022-10-20
38.3
best: 73.9 (code-davinci-002 175B (CoT))
Scaling Instruction-Finetuned Language Models arXiv:2210.11416
Multi-Task LearningonBBH-nlp
Average (%)· 2022-10-20
62.7
best: 86.3 (Qwen2.5-72B)
Scaling Instruction-Finetuned Language Models arXiv:2210.11416

Natural Language Processing4 results

Question AnsweringonStrategyQA
Accuracy· 2022-10-20
76.4
best: 90.4 (PaLM 2 (few-shot, CoT, SC))
Transcending Scaling Laws with 0.1% Extra Compute arXiv:2210.11399
Question AnsweringonMATH
Accuracy· 2022-06-29
8.8
best: 89.7 (Gemini 2.0 Flash Experimental)
Solving Quantitative Reasoning Problems with Language Models arXiv:2206.14858
Question AnsweringonMATH
Parameters (Billions)· 2022-06-29
540
Solving Quantitative Reasoning Problems with Language Models arXiv:2206.14858
Code GenerationonMBPP
Accuracy· 2022-04-05
36.8
best: 96.6 (EG-CFG (DeepSeek-V3-0324))
PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311

Knowledge Base4 results

Mathematical Question AnsweringonMATH
Accuracy· 2022-06-29
8.8
best: 89.7 (Gemini 2.0 Flash Experimental)
Solving Quantitative Reasoning Problems with Language Models arXiv:2206.14858
Mathematical Question AnsweringonMATH
Parameters (Billions)· 2022-06-29
540
Solving Quantitative Reasoning Problems with Language Models arXiv:2206.14858
Mathematical ReasoningonMATH
Accuracy· 2022-06-29
8.8
best: 89.7 (Gemini 2.0 Flash Experimental)
Solving Quantitative Reasoning Problems with Language Models arXiv:2206.14858
Mathematical ReasoningonMATH
Parameters (Billions)· 2022-06-29
540
Solving Quantitative Reasoning Problems with Language Models arXiv:2206.14858

Reasoning2 results

Math Word Problem SolvingonMATH
Accuracy· 2022-06-29
8.8
best: 89.7 (Gemini 2.0 Flash Experimental)
Solving Quantitative Reasoning Problems with Language Models arXiv:2206.14858
Math Word Problem SolvingonMATH
Parameters (Billions)· 2022-06-29
540
Solving Quantitative Reasoning Problems with Language Models arXiv:2206.14858