TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

SotA/Natural Language Processing/Question Answering

Question Answering

406 benchmarks10817 papers

Question answering can be segmented into domain-specific tasks like community question answering and knowledge-base question answering. Popular benchmark datasets for evaluation question answering systems include SQuAD, HotPotQA, bAbI, TriviaQA, WikiQA, and many others. Models for question answering are typically evaluated on metrics like EM and F1. Some recent top performing models are T5 and XLNet.

( Image credit: SQuAD )

Benchmarks

Question Answering on SQuAD2.0

F1EM

Question Answering on SQuAD1.1

F1EMExact MatchHardware BurdenOperations per network pass

Question Answering on MATH

AccuracyParameters (Billions)

Question Answering on HotpotQA

JOINT-F1ANS-F1SUP-F1ANS-EMSUP-EMJOINT-EM

Question Answering on PIQA

Accuracy

Question Answering on BoolQ

Accuracy

Question Answering on COPA

Accuracy

Question Answering on SQuAD1.1 dev

EMF1

Question Answering on Natural Questions

EMExact Match

Question Answering on TriviaQA

EMF1Exact Match

Question Answering on OpenBookQA

Accuracy

Question Answering on WebQuestions

EMExact MatchF1

Question Answering on MSRVTT-QA

AccuracyConfidence Score

Question Answering on PubMedQA

Accuracy

Question Answering on CronQuestions

Hits@1

Question Answering on EgoSchema (fullset)

Accuracy

Question Answering on ActivityNet-QA

AccuracyConfidence Score

Question Answering on MSVD-QA

AccuracyConfidence Score

Question Answering on MedQA

Accuracy

Question Answering on NExT-QA

AccuracyWUPS

Question Answering on MAWPS

Accuracy (%)

Question Answering on MultiRC

F1EM

Question Answering on SVAMP

Execution AccuracyAccuracy

Question Answering on WikiQA

MAPMRRMAP

Question Answering on SIQA

Accuracy

Question Answering on StoryCloze

Accuracy

Question Answering on TruthfulQA

MC1% true% infoMC2% true (GPT-judge)BLEURTROUGEBLEUEMAccuracy

Question Answering on DaNetQA

Accuracy

Question Answering on KILT: ELI5

F1KILT-RLR-PrecRecall@5ROUGE-LKILT-F1Rouge-L

Question Answering on TimeQuestions

P@1

Question Answering on MedMCQA

Dev Set (Acc-%)Test Set (Acc-%)

Question Answering on NewsQA

F1EM

Question Answering on CNN / Daily Mail

CNNDaily Mail

Question Answering on DROP Test

F1

Question Answering on KILT: Natural Questions

KILT-EMR-PrecRecall@5EMF1KILT-F1

Question Answering on KILT: TriviaQA

KILT-EMR-PrecRecall@5EMF1KILT-F1

Question Answering on Math23K

Accuracy (5-fold)Accuracy (training-test)weakly-supervised

Question Answering on EgoSchema (subset)

AccuracyInference Speed (s)

Question Answering on KILT: HotpotQA

KILT-EMR-PrecRecall@5EMF1KILT-F1

Question Answering on TGIF-QA

AccuracyConfidence Score

Question Answering on TrecQA

MAPMRR

Question Answering on IntentQA

Accuracy

Question Answering on SQuAD2.0 dev

F1EM

Question Answering on QALD-9-Plus

Macro F1

Question Answering on MultiTQ

Hits@1Hits@10

Question Answering on Video-MME

Accuracy (%)

Question Answering on ALG514

Accuracy (%)

Question Answering on ScienceQA

Avg. AccuracyNatural ScienceSocial ScienceLanguage ScienceText ContextImage ContextNo ContextGrades 1-6Grades 7-12

Question Answering on ASDiv-A

Execution Accuracy

Question Answering on BIG-bench (Hyperbaton)

Accuracy

Question Answering on BIG-bench (Movie Recommendation)

Accuracy

Question Answering on BIG-bench (Navigate)

Accuracy

Question Answering on BIG-bench (Ruin Names)

Accuracy

Question Answering on Bamboogle

Accuracy

Question Answering on CoQA

OverallOut-of-domainIn-domainF1-Score

Question Answering on NExT-GQA

Acc@GQA

Question Answering on NarrativeQA

Rouge-LBLEU-1BLEU-4METEOR

Question Answering on OBQA

Accuracy

Question Answering on SearchQA

EMN-gram F1Unigram AccF1

Question Answering on TIQ

P@1

Question Answering on TVQA

Accuracy

Question Answering on VNBench

Accuracy

Question Answering on Video-MME (w/o subs)

Accuracy (%)

Question Answering on WikiHop

Test

Question Answering on FEVER

EM

Question Answering on Geometry3K

Accuracy (%)

Question Answering on MMLU (College Biology)

Accuracy

Question Answering on MMLU (Medical Genetics)

Accuracy

Question Answering on Natural Questions (long)

F1EM

Question Answering on TempQuestions

F1Hits@1

Question Answering on TyDiQA-GoldP

EMF1

Question Answering on WebQuestionsSP

Hits@1F1Accuracy

Question Answering on BioASQ

Accuracy

Question Answering on FQuAD

EMF1

Question Answering on LC-QuAD 1.0

F1

Question Answering on QASent

MAPMRR

Question Answering on Quasart-T

EM

Question Answering on SQA3D

AnswerExactMatch (Question Answering)

Question Answering on Story Cloze

Accuracy

Question Answering on YahooCQA

P@1MRR

Question Answering on DROP

Accuracy

Question Answering on ELI5

Rouge-LRouge-1Rouge-2

Question Answering on FinQA

Execution AccuracyProgram Accuracy

Question Answering on FriendsQA

EMF1

Question Answering on MMLU (Professional medicine)

Accuracy

Question Answering on NExT-QA (Open-ended VideoQA)

AccuracyConfidence Score

Question Answering on NQ (BEIR)

nDCG@10

Question Answering on ParaMAWPS

Accuracy (%)

Question Answering on PeerQA

Prometheus-2 Answer CorrectnessRouge-LAlignScore

Question Answering on Quasar

EM (Quasar-T)F1 (Quasar-T)

Question Answering on RACE

RACE-mRACE-hRACE

Question Answering on ReClor

AccuracyAccuracy (easy)Accuracy (hard)

Question Answering on StrategyQA

AccuracyEM

Question Answering on ComplexWebQuestions

AccuracyEMF1Hits@1

Question Answering on ConvFinQA

Execution AccuracyProgram Accuracy

Question Answering on DRAW-1K

Accuracy (%)

Question Answering on MMLU (Abstract Algebra)

Accuracy

Question Answering on MMLU (Astronomy)

Accuracy

Question Answering on MMLU (College Chemistry)

Accuracy

Question Answering on MMLU (College Mathematics)

Accuracy

Question Answering on MMLU (College Physics)

Accuracy

Question Answering on MMLU (Econometrics)

Accuracy

Question Answering on MMLU (Electrical Engineer)

Accuracy

Question Answering on MMLU (Elementary Mathematics)

Accuracy

Question Answering on MMLU (Formal Logic)

Accuracy

Question Answering on MMLU (High School Biology)

Accuracy

Question Answering on MMLU (High School Computer Science)

Accuracy

Question Answering on MMLU (High School Mathematics)

Accuracy

Question Answering on MMLU (High School Statistics)

Accuracy

Question Answering on MathQA

Answer Accuracy

Question Answering on SemEvalCQA

P@1MAP

Question Answering on SimpleQuestionsWikiData

F1

Question Answering on AI2 Kaggle Dataset

P@1

Question Answering on BIG-bench (Novel Concepts)

Accuracy

Question Answering on BLURB

Accuracy

Question Answering on CICERO

ROUGEExact Match

Question Answering on CheGeKa

Accuracy

Question Answering on Complex-CronQuestions

Hits@1

Question Answering on EgoTaskQA

Direct

Question Answering on FairytaleQA

F1Rouge-L

Question Answering on FiQA-2018 (BEIR)

nDCG@10

Question Answering on HotpotQA (BEIR)

nDCG@10

Question Answering on HybridQA

ANS-EM

Question Answering on MMLU (College Computer Science)

Accuracy

Question Answering on MMLU (High School Chemistry)

Accuracy

Question Answering on MMLU (High School Physics)

Accuracy

Question Answering on MMLU (Machine Learning)

Accuracy

Question Answering on MS MARCO

Rouge-LBLEU-1

Question Answering on Molweni

EMF1

Question Answering on MultiQ

Accuracy

Question Answering on QuALITY

Accuracy

Question Answering on RuOpenBookQA

Accuracy

Question Answering on STAR Benchmark

Accuracy

Question Answering on Zero-shot Video Question Answering on LongVideoBench

Accuracy (% )

Question Answering on ASNQ

MAPMRR

Question Answering on CaseHOLD

Macro F1 (10-fold)

Question Answering on DuoRC

Accuracy

Question Answering on IndicGLUE WSTP Pa

Accuracy

Question Answering on MLQA

F1EM

Question Answering on MMLU (Anatomy)

Accuracy

Question Answering on MMLU (Clinical Knowledge)

Accuracy

Question Answering on MMLU (College Medicine)

Accuracy

Question Answering on Mathematics Dataset

Accuracy

Question Answering on OTT-QA

ANS-EM

Question Answering on SCDE

BAPADE

Question Answering on VNHSGE-English

Accuracy

Question Answering on XQuAD

EMF1Average F1

Question Answering on AGI Eval

Accuracy

Question Answering on Aristo Kaggle Allen AI 8th grade questions

1:1 Accuracy

Question Answering on CODAH

Accuracy

Question Answering on COMPLEXQUESTIONS

F1

Question Answering on CliCR

F1

Question Answering on CrowdSource QA

MSE

Question Answering on DuReader

EM

Question Answering on FrenchMedMCQA

Exact Match AccuracyHamming Score

Question Answering on GeoS

Accuracy (%)

Question Answering on MRQA

Average F1

Question Answering on MVBench

Accuracy

Question Answering on MapEval-API

Accuracy (%)

Question Answering on MedTurkQuAD: Medical Turkish Question-Answering Dataset

Exact MatchF1 Score

Question Answering on MuLD (HotpotQA)

BLEU-1BLEU-4METEORRouge-L

Question Answering on MuLD (NarrativeQA)

BLEU-1BLEU-4METEORRouge-L

Question Answering on NaturalQA

EMF1

Question Answering on PopQA

Accuracy

Question Answering on PubChemQA

BLEU-2BLEU-4ROUGE-1ROUGE-2ROUGE-LMEATOR

Question Answering on QuAC

F1HEQDHEQQ

Question Answering on Quora Question Pairs

Accuracy

Question Answering on SQuAD

Exact MatchF1

Question Answering on SVAMP (1:N)

Execution Accuracy

Question Answering on TQA

Exact Match

Question Answering on TempQA-WD

F1

Question Answering on Torque

F1EMC

Question Answering on TweetQA

BLEU-1ROUGE-L

Question Answering on Ubuntu Dialogue (v2, Ranking)

1 in 10 R@11 in 10 R@21 in 10 R@51 in 2 R@1

Question Answering on UniProtQA

BLEU-2BLEU-4ROUGE-1ROUGE-2ROUGE-LMEATOR

Question Answering on VNHSGE Mathematics

Accuracy

Question Answering on VNHSGE-Biology

Accuracy

Question Answering on VNHSGE-Chemistry

Accuracy

Question Answering on VNHSGE-Civic

Accuracy

Question Answering on VNHSGE-Geography

Accuracy

Question Answering on VNHSGE-History

Accuracy

Question Answering on VNHSGE-Literature

Accuracy

Question Answering on VNHSGE-Physics

Accuracy

Question Answering on WikiSQL

Exact Match (EM)

Question Answering on 60k Stack Overflow Questions

F1 Score

Question Answering on AviationQA

Hits@1

Question Answering on BBH

Accuracy

Question Answering on COCO Visual Question Answering (VQA) real images 1.0 open ended

Test

Question Answering on ChAII - Hindi and Tamil Question Answering

Jaccard

Question Answering on CinePile: A Long Video Question Answering Dataset and Benchmark

Accuracy

Question Answering on EfficientQA dev

Accuracy

Question Answering on EfficientQA test

Accuracy

Question Answering on Extended XQuAD

F1 (QE-PE)EM(QE-PE)F1 (QE-PH)EM(QE-PH)F1(QH-PE)EM(QH-PE)F1(QH-PH)EM(QH-PH)

Question Answering on GSM-Plus

1:1 Accuracy

Question Answering on GrailQA

Compositional EMCompositional F1EMI.I.D. EMI.I.D. F1Overall EMOverall F1Zero-shot EMZero-shot F1

Question Answering on GraphQuestions

Accuracy

Question Answering on HellaSwag

Accuracy

Question Answering on JD Product Question Answer

BLEU

Question Answering on JaQuAD

Exact MatchF1

Question Answering on KQA Pro

Accuracy

Question Answering on MATH minival

Accuracy

Question Answering on MML

Accuracy

Question Answering on MRQA out-of-domain

Average F1

Question Answering on MapEval-Textual

Accuracy (% )

Question Answering on MedMCQA Dev

Accuarcy

Question Answering on MetaQA

AnswerExactMatch (Question Answering)

Question Answering on MultiSpanQA

Exact F1

Question Answering on Natural Questions (short)

Exact Match

Question Answering on PEN

Accuracy (%)

Question Answering on QASPER

Token F1

Question Answering on RecipeQA

Accuracy

Question Answering on SWAG

Accuracy

Question Answering on SberQuAD

EMF1

Question Answering on SchizzoSQUAD

Average F1Averaged Precision

Question Answering on SimpleQuestions

EMF1

Question Answering on StepGame

1-of-100 Accuracy

Question Answering on TAT-QA

Exact Match (EM)

Question Answering on Ubuntu Dialogue (v1, Ranking)

1 in 10 R@11 in 10 R@21 in 10 R@51 in 2 R@1

Question Answering on WebQSP

EM

Question Answering on WebQSP-WD

Avg F1

Question Answering on WebSRC

F1

Question Answering on WikiTableQuestions

AccuracyAccuracy (Test)

Question Answering on Children's Book Test

Accuracy-CNAccuracy-NE

Question Answering on ConditionalQA

Conditional (answers)Conditional (w/ conditions)Overall (answers)Overall (w/ conditions)

Question Answering on GeoQuestions1089

Percentage correct

Question Answering on MCTest-160

Accuracy

Question Answering on MCTest-500

Accuracy

Question Answering on Reverb

Accuracy

Question Answering on bAbi

Accuracy (trained on 10k)Accuracy (trained on 1k)Mean Error Rate

Question Answering on catbAbI LM-mode

Accuracy (mean)

Question Answering on catbAbI QA-mode

1:1 Accuracy