PaLM-540B (few-shot, k=5)

Reported on 6 benchmarks across 4 tasks · 2 papers · 5 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing4 results

Question AnsweringonBIG-bench (Novel Concepts)
Accuracy· 2022-04-05
71.9
SOTA
PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311
Common Sense ReasoningonBIG-bench (Winowhy)
Accuracy· 2022-04-05
65.9
SOTA
PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311
Common Sense ReasoningonBIG-bench (Known Unknowns)
Accuracy· 2022-04-05
73.9
SOTA
PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311
MemorizationonBIG-bench (Hindu Knowledge)
Accuracy· 2022-04-05
95.4
SOTA
PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311

Methodology2 results

Logical ReasoningonBIG-bench (StrategyQA)
Accuracy· 2022-04-05
73.9
SOTA
PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311
Logical ReasoningonBIG-bench (Logic Grid Puzzle)
Accuracy· 2023-05-17
42.4
best: 44 (Chinchilla-70B (few-shot, k=5))
PaLM 2 Technical Report arXiv:2305.10403