TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

SotA/Natural Language Processing/Visual Question Answering (VQA)

Visual Question Answering (VQA)

230 benchmarks2167 papers

Visual Question Answering (VQA) is a task in computer vision that involves answering questions about an image. The goal of VQA is to teach machines to understand the content of an image and answer questions about it in natural language.

Image Source: visualqa.org

Benchmarks

Visual Question Answering (VQA) on MM-Vet

GPT-4 scoreAccParams

Visual Question Answering (VQA) on GQA Test2019

AccuracyBinaryOpenConsistencyPlausibilityValidityDistribution

Visual Question Answering (VQA) on VideoInstruct

gpt-scoremeanCorrectness of InformationDetail OrientationContextual UnderstandingTemporal UnderstandingConsistency

Visual Question Answering (VQA) on VQA v2 test-dev

Accuracy

Visual Question Answering (VQA) on VQA v2 test-std

overallyes/nonumberotherAccuracy

Visual Question Answering (VQA) on MSVD-QA

Accuracy

Visual Question Answering (VQA) on MSRVTT-QA

AccuracyTest Accuracy

Visual Question Answering (VQA) on OK-VQA

AccuracyExact Match (EM)Recall@5

Visual Question Answering (VQA) on DocVQA test

ANLSAccuracy

Visual Question Answering (VQA) on ChartQA

1:1 Accuracy

Visual Question Answering (VQA) on InfographicVQA

ANLS

Visual Question Answering (VQA) on ScanQA Test w/ objects

CIDErBLEU-4Exact MatchROUGEMETEORBLEU-1

Visual Question Answering (VQA) on GQA test-dev

Accuracy

Visual Question Answering (VQA) on CLEVR

Accuracy

Visual Question Answering (VQA) on VizWiz 2020 VQA

overallyes/nonumberotherunanswerable

Visual Question Answering (VQA) on VQA v2 val

Accuracy

Visual Question Answering (VQA) on COCO Visual Question Answering (VQA) real images 1.0 open ended

Percentage correct

Visual Question Answering (VQA) on InfiMM-Eval

Overall scoreDeductiveAbductiveAnalogicalParams

Visual Question Answering (VQA) on A-OKVQA

MC AccuracyDA VQA Score

Visual Question Answering (VQA) on SQA3D

Exact Match

Visual Question Answering (VQA) on TextVQA test-standard

overall

Visual Question Answering (VQA) on ViP-Bench

GPT-4 score (bbox)GPT-4 score (human)

Visual Question Answering (VQA) on IconQA

Sub-tasks (Img.)Sub-tasks (Txt.)Sub-tasks (Blank)Reasoning (Geo.)Reasoning (Cou.)Reasoning (Com.)Reasoning (Spa.)Reasoning (Sce.)Reasoning (Pat.)Reasoning (Tim.)Reasoning (Fra.)Reasoning (Est.)Reasoning (Alg.)Reasoning (Mea.)Reasoning (Sen.)Reasoning (Pro.)

Visual Question Answering (VQA) on VCR (Q-A) test

Accuracy

Visual Question Answering (VQA) on BenchLMM

GPT-3.5 score

Visual Question Answering (VQA) on COCO Visual Question Answering (VQA) real images 1.0 multiple choice

Percentage correct

Visual Question Answering (VQA) on VQA-CP

Score

Visual Question Answering (VQA) on VizWiz 2018

overallyes/nonumberotherunanswerable

Visual Question Answering (VQA) on VLM2-Bench

GC-matGC-trkOC-cprOC-cntOC-grpPC-cprPC-cntPC-grpPC-VIDAverage Score on VLM2-bench (9 subtasks)

Visual Question Answering (VQA) on VQA-CE

Accuracy (Counterexamples)

Visual Question Answering (VQA) on VCR (QA-R) test

Accuracy

Visual Question Answering (VQA) on 6-DoF SpatialBench

TotalPosition-relPosition-absOrientation-relOrientation-abs

Visual Question Answering (VQA) on GQA test-std

Accuracy

Visual Question Answering (VQA) on IllusionVQA

Accuracy

Visual Question Answering (VQA) on InfoSeek

Accuracy

Visual Question Answering (VQA) on SME

BLEU-4METEORROUGE-LCIDErSPICEDetectionACC#Learning Samples (N)

Visual Question Answering (VQA) on VCR (Q-AR) test

Accuracy

Visual Question Answering (VQA) on VQA v1 test-dev

Accuracy

Visual Question Answering (VQA) on PlotQA

1:1 Accuracy

Visual Question Answering (VQA) on PlotQA-D1

1:1 Accuracy

Visual Question Answering (VQA) on PlotQA-D2

1:1 Accuracy

Visual Question Answering (VQA) on VQA v1 test-std

Accuracy

Visual Question Answering (VQA) on VizWiz 2020 Answerability

average_precisionf1_score

Visual Question Answering (VQA) on WHOOPS!

Exact MatchBEM

Visual Question Answering (VQA) on 3D MM-Vet

Overall Accuracy

Visual Question Answering (VQA) on AutoHallusion

Overall Accuracy

Visual Question Answering (VQA) on CLEVR-Humans

Accuracy

Visual Question Answering (VQA) on COCO Visual Question Answering (VQA) real images 2.0 open ended

Percentage correct

Visual Question Answering (VQA) on EmbSpatial-Bench

Generation

Visual Question Answering (VQA) on GQA-REX

BLEU-4CIDErGQA-testGQA-valGroundingMETEORROUGE-LSPICE

Visual Question Answering (VQA) on MMBench

GPT-3.5 score

Visual Question Answering (VQA) on QLEVR

Overall Accuracy

Visual Question Answering (VQA) on RealCQA

1:1 Accuracy

Visual Question Answering (VQA) on V*bench

Accuracy

Visual Question Answering (VQA) on AI2D

EM

Visual Question Answering (VQA) on COCO Visual Question Answering (VQA) abstract 1.0 multiple choice

Percentage correct

Visual Question Answering (VQA) on COCO Visual Question Answering (VQA) abstract images 1.0 open ended

Percentage correct

Visual Question Answering (VQA) on PMC-VQA

AccuracyBLEU-1

Visual Question Answering (VQA) on Visual7W

Percentage correct

Visual Question Answering (VQA) on DREAM

Accuracy

Visual Question Answering (VQA) on F-VQA

Top-1 AccuracyTop-3 AccuracyAccuracyMRMRR

Visual Question Answering (VQA) on FigureQA - test 1

1:1 Accuracy

Visual Question Answering (VQA) on GQA

Accuracy

Visual Question Answering (VQA) on GRIT

VQA (ablation)VQA (test)

Visual Question Answering (VQA) on HallusionBench

Question Pair Acc Question Pair Acc

Visual Question Answering (VQA) on ReClor

AccuracyAccuracy (easy)Accuracy (hard)

Visual Question Answering (VQA) on VCR (Q-A) dev

Accuracy

Visual Question Answering (VQA) on VCR (Q-AR) dev

Accuracy

Visual Question Answering (VQA) on VCR (QA-R) dev

Accuracy

Visual Question Answering (VQA) on MMHal-Bench

Hallucination RateScore

Visual Question Answering (VQA) on TDIUC

Accuracy

Visual Question Answering (VQA) on TGIF-QA

Accuracy

Visual Question Answering (VQA) on VQA v2

Accuracy

Visual Question Answering (VQA) on VQA-X

Accuracy

Visual Question Answering (VQA) on AID-VQA

Acc. (test)

Visual Question Answering (VQA) on AMBER

AccuracyF1

Visual Question Answering (VQA) on ActivityNet

ClipMatch@1ClipMatch@5ContainsExactMatchFollow-up ClipMatch@1Follow-up ClipMatch@5Follow-up ContainsFollow-up ExactMatch

Visual Question Answering (VQA) on ArtQuest

1:1 Accuracy

Visual Question Answering (VQA) on BIOMRC

Acc

Visual Question Answering (VQA) on COCO

ClipMatch@1ClipMatch@5ContainsExactMatch

Visual Question Answering (VQA) on CORE-MM

AbductiveAnalogicalDeductiveOverall scoreParams

Visual Question Answering (VQA) on DVQA test-familiar

1:1 Accuracy

Visual Question Answering (VQA) on DeepForm

F1

Visual Question Answering (VQA) on DocVQA

ANLS

Visual Question Answering (VQA) on DocVQA val

Accuracybk lôn

Visual Question Answering (VQA) on EarthVQA

Overall Accuracy

Visual Question Answering (VQA) on EgoSchema

Acc

Visual Question Answering (VQA) on ImageNet

ClipMatch@1ClipMatch@5ContainsExactMatchFollow-up ClipMatch@1Follow-up ClipMatch@5Follow-up ContainsFollow-up ExactMatch

Visual Question Answering (VQA) on MME

Acc

Visual Question Answering (VQA) on MVBench

Acc

Visual Question Answering (VQA) on MapEval-Visual

Accuracy (% )

Visual Question Answering (VQA) on OVAD benchmark

Contains w. SynonymsExactMatch w. Synonyms

Visual Question Answering (VQA) on RSVQA-HR

zero-shot Acc

Visual Question Answering (VQA) on RetVQA

AccuarcyAccuracy * Fluency

Visual Question Answering (VQA) on SIRI-WHU

Acc. (test)

Visual Question Answering (VQA) on TextVQA

Acc

Visual Question Answering (VQA) on UQuAD

Exact MatchF1

Visual Question Answering (VQA) on Video MME

Acc

Visual Question Answering (VQA) on Visual Genome (pairs)

Percentage correct

Visual Question Answering (VQA) on Visual Genome (subjects)

Percentage correct

Visual Question Answering (VQA) on VisualMRC

CIDEr

Visual Question Answering (VQA) on VizWiz

Accuracy

Visual Question Answering (VQA) on VizWiz 2018 Answerability

average_precisionf1_score

Visual Question Answering (VQA) on WebSRC

EM

Visual Question Answering (VQA) on ZS-F-VQA

Top-1 Accuracy

Visual Question Answering (VQA) on MM-Vet (w/o External Tools)

GPT-4 score

Visual Question Answering (VQA) on MM-Vet v2

GPT-4 scoreParams