TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/PaLM 2-M (one-shot)

PaLM 2-M (one-shot)

Reported on 14 benchmarks across 6 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing12 results

  • Question AnsweringonNatural Questions
    EM· 2023-05-17
    32
    best: 64 (Atlas (full, Wiki-dec-2018 index))
    PaLM 2 Technical ReportarXiv:2305.10403
  • Question AnsweringonStory Cloze
    Accuracy· 2023-05-17
    86.7
    best: 87.8 (Neo-6B (QA + WS))
    PaLM 2 Technical ReportarXiv:2305.10403
  • Question AnsweringonMultiRC
    F1· 2023-05-17
    84.1
    best: 90.1 (PaLM 540B (finetuned) )
    PaLM 2 Technical ReportarXiv:2305.10403
  • Question AnsweringonWebQuestions
    EM· 2023-05-17
    26.9
    best: 84.6 (PoG-GPT4 (Tan et al., 2024))
    PaLM 2 Technical ReportarXiv:2305.10403
  • Question AnsweringonTriviaQA
    EM· 2023-05-17
    81.7
    best: 87.5 (Claude 2 (few-shot, k=5))
    PaLM 2 Technical ReportarXiv:2305.10403
  • Question AnsweringonTyDiQA-GoldP
    F1· 2023-05-17
    73.3
    best: 88.5 (U-PaLM 62B (fine-tuned))
    PaLM 2 Technical ReportarXiv:2305.10403
  • Common Sense ReasoningonReCoRD
    F1· 2023-05-17
    92.4
    best: 96.4 (Turing NLR v5 XXL 5.4B (fine-tuned))
    PaLM 2 Technical ReportarXiv:2305.10403
  • Word Sense DisambiguationonWords in Context
    Accuracy· 2023-05-17
    52
    best: 85.3 (COSINE + Transductive Learning)
    PaLM 2 Technical ReportarXiv:2305.10403
  • Natural Language InferenceonANLI test
    A1· 2023-05-17
    58.1
    best: 81.8 (T5-3B (explanation prompting))
    PaLM 2 Technical ReportarXiv:2305.10403
  • Natural Language InferenceonANLI test
    A2· 2023-05-17
    49.5
    best: 72.5 (T5-3B (explanation prompting))
    PaLM 2 Technical ReportarXiv:2305.10403
  • Natural Language InferenceonANLI test
    A3· 2023-05-17
    54.5
    best: 74.8 (T5-3B (explanation prompting))
    PaLM 2 Technical ReportarXiv:2305.10403
  • Natural Language InferenceonCommitmentBank
    Accuracy· 2023-05-17
    80.4
    best: 100 (PaLM 540B (finetuned))
    PaLM 2 Technical ReportarXiv:2305.10403

Medical1 result

  • Language ModellingonLAMBADA
    Accuracy· 2023-05-17
    83.7
    best: 89.7 (PaLM-540B (Few-Shot))
    PaLM 2 Technical ReportarXiv:2305.10403

Knowledge Base1 result

  • Text SummarizationonX-Sum
    ROUGE-2· 2023-05-17
    17.2
    best: 26.7 (Selfmem)
    PaLM 2 Technical ReportarXiv:2305.10403