Visual Question Answering (VQA) on A-OKVQA

Metric: MC Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	MC Accuracy▼	Extra Data	Paper	Date↕	Code
1	SMoLA-PaLI-X Specialist Model	83.75	Yes	Omni-SMoLA: Boosting Generalist Multimodal Model...	2023-12-01	-
2	PaLI-X-VPD	80.4	No	Visual Program Distillation: Distilling Tools an...	2023-12-05	-
3	Prophet	75.1	No	Prophet: Prompting Large Language Models with Co...	2023-03-03	Code
4	PromptCap	73.2	No	PromptCap: Prompt-Guided Task-Aware Image Captio...	2022-11-15	Code
5	MC-CoT	71	No	Boosting the Power of Small Multimodal Reasoning...	2023-11-23	Code
6	HYDRA	56.35	No	HYDRA: A Hyper Agent for Dynamic Compositional V...	2024-03-19	Code
7	GPV-2	53.7	No	Webly Supervised Concept Expansion for General P...	2022-02-04	-
8	KRISP	42.2	No	KRISP: Integrating Implicit and Symbolic Knowled...	2020-12-20	-
9	ViLBERT - VQA	42.1	No	ViLBERT: Pretraining Task-Agnostic Visiolinguist...	2019-08-06	Code
10	LXMERT	41.6	No	LXMERT: Learning Cross-Modality Encoder Represen...	2019-08-20	Code
11	ViLBERT	41.5	No	ViLBERT: Pretraining Task-Agnostic Visiolinguist...	2019-08-06	Code
12	Pythia	40.1	No	Pythia v0.1: the Winning Entry to the VQA Challe...	2018-07-26	Code
13	ViLBERT - OK-VQA	34.1	No	ViLBERT: Pretraining Task-Agnostic Visiolinguist...	2019-08-06	Code

#1SMoLA-PaLI-X Specialist ModelSOTA
83.75
MC Accuracy· Extra Data· 2023-12-01
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts
#2PaLI-X-VPD
80.4
MC Accuracy· 2023-12-05
Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models
#3ProphetSOTA
75.1
MC Accuracy· 2023-03-03
Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering Code
#4PromptCapSOTA
73.2
MC Accuracy· 2022-11-15
PromptCap: Prompt-Guided Task-Aware Image Captioning Code
#5MC-CoT
71
MC Accuracy· 2023-11-23
Boosting the Power of Small Multimodal Reasoning Models to Match Larger Models with Self-Consistency Training Code
#6HYDRA
56.35
MC Accuracy· 2024-03-19
HYDRA: A Hyper Agent for Dynamic Compositional Visual Reasoning Code
#7GPV-2SOTA
53.7
MC Accuracy· 2022-02-04
Webly Supervised Concept Expansion for General Purpose Vision Models
#8KRISPSOTA
42.2
MC Accuracy· 2020-12-20
KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA
#9ViLBERT - VQASOTA
42.1
MC Accuracy· 2019-08-06
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Code
#10LXMERT
41.6
MC Accuracy· 2019-08-20
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Code
#11ViLBERT
41.5
MC Accuracy· 2019-08-06
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Code
#12PythiaSOTA
40.1
MC Accuracy· 2018-07-26
Pythia v0.1: the Winning Entry to the VQA Challenge 2018 Code
#13ViLBERT - OK-VQA
34.1
MC Accuracy· 2019-08-06
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Code