UL2 20B (chain-of-thought + self-consistency)

Reported on 3 benchmarks across 1 task · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing3 results

Common Sense ReasoningonARC (Challenge)
Accuracy· 2022-05-10
49.5
best: 96.4 (GPT-4 (few-shot, k=25))
UL2: Unifying Language Learning Paradigms arXiv:2205.05131
Common Sense ReasoningonARC (Easy)
Accuracy· 2022-05-10
69.8
best: 95.2 (ST-MoE-32B 269B (fine-tuned))
UL2: Unifying Language Learning Paradigms arXiv:2205.05131
Common Sense ReasoningonCommonsenseQA
Accuracy· 2022-05-10
55.7
best: 92.54 (GPT-4o (HPT))
UL2: Unifying Language Learning Paradigms arXiv:2205.05131