Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/PaLM 2 (few-shot, k=3, Direct)

PaLM 2 (few-shot, k=3, Direct)

Reported on 13 benchmarks across 4 tasks · 1 paper · 4 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing9 results

Question AnsweringonBIG-bench (Ruin Names)
Accuracy· 2023-05-17
90
SOTA
PaLM 2 Technical Report arXiv:2305.10403
Common Sense ReasoningonBIG-bench (Causal Judgment)
Accuracy· 2023-05-17
62
SOTA
PaLM 2 Technical Report arXiv:2305.10403
Common Sense ReasoningonBIG-bench (Disambiguation QA)
Accuracy· 2023-05-17
78.8
SOTA
PaLM 2 Technical Report arXiv:2305.10403
Question AnsweringonBIG-bench (Movie Recommendation)
Accuracy· 2023-05-17
93.6
best: 94.4 (PaLM 2 (few-shot, k=3, CoT))
PaLM 2 Technical Report arXiv:2305.10403
Question AnsweringonBIG-bench (Navigate)
Accuracy· 2023-05-17
68.8
best: 91.2 (PaLM 2 (few-shot, k=3, CoT))
PaLM 2 Technical Report arXiv:2305.10403
Question AnsweringonBIG-bench (Hyperbaton)
Accuracy· 2023-05-17
84.8
best: 92 (Bloomberg GPT (few-shot, k=3))
PaLM 2 Technical Report arXiv:2305.10403
Common Sense ReasoningonBIG-bench (Sports Understanding)
Accuracy· 2023-05-17
90.8
best: 98 (PaLM 2(few-shot, k=3, CoT))
PaLM 2 Technical Report arXiv:2305.10403
Common Sense ReasoningonBIG-bench (Date Understanding)
Accuracy· 2023-05-17
74
best: 91.2 (PaLM 2 (few-shot, k=3, CoT))
PaLM 2 Technical Report arXiv:2305.10403
Sarcasm DetectiononBIG-bench (SNARKS)
Accuracy· 2023-05-17
78.7
best: 84.8 (PaLM 2(few-shot, k=3, CoT))
PaLM 2 Technical Report arXiv:2305.10403

Methodology4 results

Logical ReasoningonBIG-bench (Formal Fallacies Syllogisms Negation)
Accuracy· 2023-05-17
64.8
SOTA
PaLM 2 Technical Report arXiv:2305.10403
Logical ReasoningonBIG-bench (Penguins In A Table)
Accuracy· 2023-05-17
65.8
best: 84.9 (PaLM 2 (few-shot, k=3, CoT))
PaLM 2 Technical Report arXiv:2305.10403
Logical ReasoningonBIG-bench (Temporal Sequences)
Accuracy· 2023-05-17
96.4
best: 100 (PaLM 2 (few-shot, k=3, CoT))
PaLM 2 Technical Report arXiv:2305.10403
Logical ReasoningonBIG-bench (Reasoning About Colored Objects)
Accuracy· 2023-05-17
61.2
best: 91.2 (PaLM 2 (few-shot, k=3, CoT))
PaLM 2 Technical Report arXiv:2305.10403