Codex 5-shot CoT

Natural Language Processing4 results

• Question AnsweringonPubMedQA

  Accuracy· 2022-07-17

  78.2

  best: 81.6 (Meditron-70B (CoT + SC))

  SOTA
  Can large language models reason about medical questions?arXiv:2207.08143
• Question AnsweringonMedQA

  Accuracy· 2022-07-17

  60.2

  best: 91.1 (Med-Gemini)

  SOTA
  Can large language models reason about medical questions?arXiv:2207.08143
• Question AnsweringonMedMCQA

  Dev Set (Acc-%)· 2022-07-17

  0.597

  best: 66 (Meditron-70B (CoT + SC))

  SOTA
  Can large language models reason about medical questions?arXiv:2207.08143
• Question AnsweringonMedMCQA

  Test Set (Acc-%)· 2022-07-17

  0.627

  best: 0.723 (Med-PaLM 2 (ER))

  SOTA
  Can large language models reason about medical questions?arXiv:2207.08143