Pythia 6.9B (0-shot)

Reported on 4 benchmarks across 4 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing3 results

Question AnsweringonPIQA
Accuracy· 2023-04-03
75.2
best: 90.1 (Unicorn 11B (fine-tuned))
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling arXiv:2304.01373
Common Sense ReasoningonWinoGrande
Accuracy· 2023-04-03
60.9
best: 96.1 (ST-MoE-32B 269B (fine-tuned))
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling arXiv:2304.01373
Coreference ResolutiononWinograd Schema Challenge
Accuracy· 2023-04-03
36.5
best: 100 (PaLM 540B (fine-tuned))
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling arXiv:2304.01373

Medical1 result

Language ModellingonLAMBADA
Accuracy· 2023-04-03
67.28
best: 89.7 (PaLM-540B (Few-Shot))
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling arXiv:2304.01373