TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/Chinchilla (few-shot, k=5)

Chinchilla (few-shot, k=5)

Reported on 19 benchmarks across 1 task · 1 paper · 11 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing19 results

  • Question AnsweringonMMLU (College Biology)
    Accuracy· 2022-11-16
    79.9
    best: 95.8 (Med-PaLM 2 (ER))
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (Machine Learning)
    Accuracy· 2022-11-16
    41.1
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (High School Physics)
    Accuracy· 2022-11-16
    36.4
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (College Chemistry)
    Accuracy· 2022-11-16
    51
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (College Computer Science)
    Accuracy· 2022-11-16
    51
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (Astronomy)
    Accuracy· 2022-11-16
    73
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (High School Biology)
    Accuracy· 2022-11-16
    80.3
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (High School Chemistry)
    Accuracy· 2022-11-16
    58.1
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (Elementary Mathematics)
    Accuracy· 2022-11-16
    41.5
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (High School Statistics)
    Accuracy· 2022-11-16
    58.8
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (College Physics)
    Accuracy· 2022-11-16
    46.1
    SOTA
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (Econometrics)
    Accuracy· 2022-11-16
    38.6
    best: 43 (Gopher (few-shot, k=5))
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (Medical Genetics)
    Accuracy· 2022-11-16
    69
    best: 92 (Med-PaLM 2 (ER))
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (High School Computer Science)
    Accuracy· 2022-11-16
    58
    best: 70 (GAL 120B (zero-shot))
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (Electrical Engineer)
    Accuracy· 2022-11-16
    62.1
    best: 62.8 (GAL 120B (zero-shot))
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (Formal Logic)
    Accuracy· 2022-11-16
    33.3
    best: 35.7 (Gopher (few-shot, k=5))
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (High School Mathematics)
    Accuracy· 2022-11-16
    31.9
    best: 32.6 (GAL 120B (zero-shot))
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (Abstract Algebra)
    Accuracy· 2022-11-16
    31
    best: 33.3 (GAL 30B (zero-shot))
    Galactica: A Large Language Model for SciencearXiv:2211.09085
  • Question AnsweringonMMLU (College Mathematics)
    Accuracy· 2022-11-16
    32
    best: 43 (GAL 120B (zero-shot))
    Galactica: A Large Language Model for SciencearXiv:2211.09085