Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/GPT-4

GPT-4

Reported on 31 benchmarks across 18 tasks · 14 papers · 16 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing18 results

Bias Detectiononrt-inod-bias
Best-of· 2024-04-15
0.5
SOTA
Benchmarking Llama2, Mistral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for Hallucinations arXiv:2404.09785
Question AnsweringonGSM-Plus
1:1 Accuracy· 2024-02-29
85.6
SOTA
GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers arXiv:2402.19255
Code GenerationonTurbulence
CorrSc· 2023-12-22
0.848
SOTA
Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Code arXiv:2312.14856
Question AnsweringonMedQA
Accuracy· uses extra data· 2023-11-28
90.2
best: 91.1 (Med-Gemini)
SOTA
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine arXiv:2311.16452
Instruction FollowingonIFEval
Inst-level loose-accuracy· 2023-11-14
85.37
best: 90.4 (AutoIF (Llama3 70B))
SOTA
Instruction-Following Evaluation for Large Language Models arXiv:2311.07911
Instruction FollowingonIFEval
Inst-level strict-accuracy· 2023-11-14
83.57
best: 86.7 (AutoIF (Llama3 70B))
SOTA
Instruction-Following Evaluation for Large Language Models arXiv:2311.07911
Instruction FollowingonIFEval
Prompt-level loose-accuracy· 2023-11-14
79.3
best: 85.6 (AutoIF (Llama3 70B))
SOTA
Instruction-Following Evaluation for Large Language Models arXiv:2311.07911
Instruction FollowingonIFEval
Prompt-level strict-accuracy· 2023-11-14
76.89
best: 80.2 (AutoIF (Llama3 70B))
SOTA
Instruction-Following Evaluation for Large Language Models arXiv:2311.07911
Legal ReasoningonLegalBench (Rule-recall)
Balanced Accuracy· 2023-03-15
59.2
SOTA
GPT-4 Technical Report arXiv:2303.08774
Sentence OrderingonEconLogicQA
Accuracy· 2024-05-13
0.5538
best: 0.5692 (GPT-4-Turbo)
EconLogicQA: A Question-Answering Benchmark for Evaluating Large Language Models in Economic Sequential Reasoning arXiv:2405.07938
Dialogue Understandingonrt-inod-jailbreaking
Best-of· 2024-04-15
0.91
best: 0.92 (Baseline)
Benchmarking Llama2, Mistral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for Hallucinations arXiv:2404.09785
Question AnsweringonTimeQuestions
P@1· 2023-09-22
30.6
best: 78.1 (TimeR4)
OpenAi's GPT4 as coding assistant arXiv:2309.12732
Visual Question Answering (VQA)onDocVQA test
ANLS· 2023-06-01
0.884
best: 0.9436 (Human)
Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering arXiv:2306.00526
Question AnsweringonMATH
Accuracy· 2023-03-22
42.5
best: 89.7 (Gemini 2.0 Flash Experimental)
Sparks of Artificial General Intelligence: Early experiments with GPT-4 arXiv:2303.12712
Question AnsweringonBLURB
Accuracy
80.56
best: 83.5 (BioLinkBERT (large))
Question AnsweringonBioASQ
Accuracy
85.71
best: 94.8 (BioLinkBERT (large))
Named Entity Recognition (NER)onNCBI-disease
F1
65.98
best: 89.71 (BioBERT)
Legal ReasoningonLegalBench (Issue-spotting)
Balanced Accuracy
82.9

Reasoning4 results

Multimodal ReasoningonAlgoPuzzleVQA
Acc· 2024-03-06
30.3
SOTA
Are Language Models Puzzle Prodigies? Algorithmic Puzzles Unveil Serious Challenges in Multimodal Reasoning arXiv:2403.03864
Math Word Problem SolvingonGSM-Plus
1:1 Accuracy· 2024-02-29
85.6
SOTA
GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers arXiv:2402.19255
Math Word Problem SolvingonMATH
Accuracy· 2023-03-22
42.5
best: 89.7 (Gemini 2.0 Flash Experimental)
Sparks of Artificial General Intelligence: Early experiments with GPT-4 arXiv:2303.12712
Arithmetic ReasoningonGSM8K
Accuracy· 2023-03-22
87.1
best: 97.72 (Claude 3.5 Sonnet (HPT))
Sparks of Artificial General Intelligence: Early experiments with GPT-4 arXiv:2303.12712

Knowledge Base4 results

Mathematical Question AnsweringonGSM-Plus
1:1 Accuracy· 2024-02-29
85.6
SOTA
GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers arXiv:2402.19255
Mathematical ReasoningonGSM-Plus
1:1 Accuracy· 2024-02-29
85.6
SOTA
GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers arXiv:2402.19255
Mathematical Question AnsweringonMATH
Accuracy· 2023-03-22
42.5
best: 89.7 (Gemini 2.0 Flash Experimental)
Sparks of Artificial General Intelligence: Early experiments with GPT-4 arXiv:2303.12712
Mathematical ReasoningonMATH
Accuracy· 2023-03-22
42.5
best: 89.7 (Gemini 2.0 Flash Experimental)
Sparks of Artificial General Intelligence: Early experiments with GPT-4 arXiv:2303.12712

Speech3 results

Arabic Text DiacritizationonCATT
WER (%)· uses extra data· 2024-07-03
38.311
best: 34.191 (CATT ED)
SOTA
CATT: Character-based Arabic Tashkeel Transformer arXiv:2407.03236
Arabic Text DiacritizationonCATT
DER(%)· uses extra data· 2024-07-03
9.515
best: 58.313 (Deep Diacritization (D3))
CATT: Character-based Arabic Tashkeel Transformer arXiv:2407.03236
Dialogueonrt-inod-jailbreaking
Best-of· 2024-04-15
0.91
best: 0.92 (Baseline)
Benchmarking Llama2, Mistral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for Hallucinations arXiv:2404.09785

Methodology2 results

ClassificationonMedSecId
1 shot Micro-F1· 2024-04-25
96.86
SOTA
LLM-Based Section Identifiers Excel on Open Source but Stumble in Real World Applications arXiv:2404.16294
ClassificationonMedSecId
1 shot Micro-F1· 2024-04-25
96.24
best: 96.86
LLM-Based Section Identifiers Excel on Open Source but Stumble in Real World Applications arXiv:2404.16294

Adversarial1 result

Text GenerationonHarmfulQA
ASR· 2023-08-18
65.1
SOTA
Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment arXiv:2308.09662