Visual Question Answering (VQA) on VQA v2 test-dev

Metric: Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	Accuracy▼	Extra Data	Paper	Date↕	Code
1	PaLI	84.3	No	PaLI: A Jointly-Scaled Multilingual Language-Ima...	2022-09-14	Code
2	BEiT-3	84.19	No	Image as a Foreign Language: BEiT Pretraining fo...	2022-08-22	Code
3	VLMo	82.78	No	VLMo: Unified Vision-Language Pre-Training with ...	2021-11-03	Code
4	ONE-PEACE	82.6	No	ONE-PEACE: Exploring One General Representation ...	2023-05-18	Code
5	mPLUG (Huge)	82.43	No	mPLUG: Effective and Efficient Vision-Language L...	2022-05-24	Code
6	BLIP-2 ViT-G OPT 6.7B (fine-tuned)	82.3	No	BLIP-2: Bootstrapping Language-Image Pre-trainin...	2023-01-30	Code
7	CoCa	82.3	No	CoCa: Contrastive Captioners are Image-Text Foun...	2022-05-04	Code
8	CuMo-7B	82.2	Yes	CuMo: Scaling Multimodal LLM with Co-Upcycled Mi...	2024-05-09	Code
9	OFA	82	No	OFA: Unifying Architectures, Tasks, and Modaliti...	2022-02-07	Code
10	X2-VLM (large)	81.9	No	X$^2$-VLM: All-In-One Pre-trained Model For Visi...	2022-11-22	Code
11	BLIP-2 ViT-G OPT 2.7B (fine-tuned)	81.74	No	BLIP-2: Bootstrapping Language-Image Pre-trainin...	2023-01-30	Code
12	BLIP-2 ViT-G FlanT5 XL (fine-tuned)	81.66	No	BLIP-2: Bootstrapping Language-Image Pre-trainin...	2023-01-30	Code
13	MMU	81.26	No	Achieving Human Parity on Visual Question Answer...	2021-11-17	-
14	Lyrics	81.2	No	Lyrics: Boosting Fine-grained Language-Vision Al...	2023-12-08	-
15	InternVL-C	81.2	No	InternVL: Scaling up Vision Foundation Models an...	2023-12-21	Code
16	mPLUG-2	81.11	No	mPLUG-2: A Modularized Multi-modal Foundation Mo...	2023-02-01	Code
17	X2-VLM (base)	80.4	No	X$^2$-VLM: All-In-One Pre-trained Model For Visi...	2022-11-22	Code
18	XFM (base)	80.4	No	Toward Building General Foundation Models for La...	2023-01-12	Code
19	VAST	80.23	Yes	-	-	-
20	Florence	80.16	No	Florence: A New Foundation Model for Computer Vi...	2021-11-22	Code
21	SimVLM	80.03	No	SimVLM: Simple Visual Language Model Pretraining...	2021-08-24	Code
22	VALOR	78.46	Yes	VALOR: Vision-Audio-Language Omni-Perception Pre...	2023-04-17	Code
23	Prismer	78.43	No	Prismer: A Vision-Language Model with Multi-Task...	2023-03-04	Code
24	X-VLM (base)	78.22	No	Multi-Grained Vision Language Pre-Training: Alig...	2021-11-16	Code
25	VK-OOD	77.9	No	-	-	Code
26	Aurora (ours, r=64)	77.69	No	-	-	-
27	VK-OOD	76.8	No	Differentiable Outlier Detection Enable Robust D...	2023-02-11	Code
28	ALBEF (14M)	75.84	No	Align before Fuse: Vision and Language Represent...	2021-07-16	Code
29	Oscar	73.82	No	Oscar: Object-Semantics Aligned Pre-training for...	2020-04-13	Code
30	UNITER (Large)	73.24	No	UNITER: UNiversal Image-TExt Representation Lear...	2019-09-25	Code
31	X-101 grid features + MCAN	72.59	No	In Defense of Grid Features for Visual Question ...	2020-01-10	Code
32	CFR	72.5	No	Coarse-to-Fine Reasoning for Visual Question Ans...	2021-10-06	Code
33	VL-BERTLARGE	71.79	No	VL-BERT: Pre-training of Generic Visual-Linguist...	2019-08-22	Code
34	ViLT-B/32	71.26	No	ViLT: Vision-and-Language Transformer Without Co...	2021-02-05	Code
35	MCAN+VC	71.21	No	Visual Commonsense R-CNN	2020-02-27	Code
36	VL-BERTBASE	71.16	No	VL-BERT: Pre-training of Generic Visual-Linguist...	2019-08-22	Code
37	VisualBERT	70.8	No	VisualBERT: A Simple and Performant Baseline for...	2019-08-09	Code
38	LXMERT (low-magnitude pruning)	70.72	No	LXMERT Model Compression for Visual Question Ans...	2023-10-23	Code
39	MCANed-6	70.63	No	Deep Modular Co-Attention Networks for Visual Qu...	2019-06-25	Code
40	ViLBERT	70.55	No	ViLBERT: Pretraining Task-Agnostic Visiolinguist...	2019-08-06	Code
41	BAN+Glove+Counter	70.04	No	Bilinear Attention Networks	2018-05-21	Code
42	LXMERT (Pre-train + scratch)	69.9	No	LXMERT: Learning Cross-Modality Encoder Represen...	2019-08-20	Code
43	Image features from bottom-up attention (adaptive K, ensemble)	69.87	No	Tips and Tricks for Visual Question Answering: L...	2017-08-09	Code
44	Pythia v0.3 + LoRRA	69.21	No	Towards VQA Models That Can Read	2019-04-18	Code
45	DMN	68.09	No	Learning to Count Objects in Natural Images for ...	2018-02-15	Code
46	LaKo	68.07	No	LaKo: Knowledge-driven Visual Question Answering...	2022-07-26	Code
47	MuRel	68.03	No	MUREL: Multimodal Relational Reasoning for Visua...	2019-02-25	Code
48	BLOCK	67.58	No	BLOCK: Bilinear Superdiagonal Fusion for Visual ...	2019-01-31	Code
49	MUTAN	67.42	No	MUTAN: Multimodal Tucker Fusion for Visual Quest...	2017-05-18	Code
50	BAN2-CTI	67.4	No	Compact Trilinear Interaction for Visual Questio...	2019-09-26	Code
51	2D continuous softmax	65.96	No	Sparse and Continuous Attention Mechanisms	2020-06-12	Code
52	BLIP-2 ViT-G FlanT5 XXL (zero-shot)	65	No	BLIP-2: Bootstrapping Language-Image Pre-trainin...	2023-01-30	Code
53	N2NMN (ResNet-152, policy search)	64.9	No	Learning to Reason: End-to-End Module Networks f...	2017-04-18	Code
54	PNP-VQA	64.8	No	Plug-and-Play VQA: Zero-shot VQA by Conjoining L...	2022-10-17	Code
55	MCB	64.7	No	Multimodal Compact Bilinear Pooling for Visual Q...	2016-06-06	Code
56	RUBi	63.18	No	RUBi: Reducing Unimodal Biases in Visual Questio...	2019-06-24	Code
57	BLIP-2 ViT-G FlanT5 XL (zero-shot)	63	No	BLIP-2: Bootstrapping Language-Image Pre-trainin...	2023-01-30	Code
58	BLIP-2 ViT-L FlanT5 XL (zero-shot)	62.3	No	BLIP-2: Bootstrapping Language-Image Pre-trainin...	2023-01-30	Code
59	Flamingo 80B	56.3	No	Flamingo: a Visual Language Model for Few-Shot L...	2022-04-29	Code
60	LocVLM-L	56.2	No	Learning to Localize Objects Improves Spatial Re...	2024-04-11	Code
61	BLIP-2 ViT-G OPT 6.7B (zero-shot)	52.6	No	BLIP-2: Bootstrapping Language-Image Pre-trainin...	2023-01-30	Code
62	BLIP-2 ViT-G OPT 2.7B (zero-shot)	52.3	No	BLIP-2: Bootstrapping Language-Image Pre-trainin...	2023-01-30	Code
63	Flamingo 9B	51.8	No	Flamingo: a Visual Language Model for Few-Shot L...	2022-04-29	Code
64	KOSMOS-1 1.6B (zero-shot)	51	No	-	-	-
65	BLIP-2 ViT-L OPT 2.7B (zero-shot)	49.7	No	BLIP-2: Bootstrapping Language-Image Pre-trainin...	2023-01-30	Code
66	Flamingo 3B	49.2	No	Flamingo: a Visual Language Model for Few-Shot L...	2022-04-29	Code
67	VLKD	44.5	No	-	-	-

#1PaLISOTA
84.3
Accuracy· 2022-09-14
PaLI: A Jointly-Scaled Multilingual Language-Image Model Code
#2BEiT-3SOTA
84.19
Accuracy· 2022-08-22
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Code
#3VLMoSOTA
82.78
Accuracy· 2021-11-03
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts Code
#4ONE-PEACE
82.6
Accuracy· 2023-05-18
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Code
#5mPLUG (Huge)
82.43
Accuracy· 2022-05-24
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections Code
#6BLIP-2 ViT-G OPT 6.7B (fine-tuned)
82.3
Accuracy· 2023-01-30
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Code
#7CoCa
82.3
Accuracy· 2022-05-04
CoCa: Contrastive Captioners are Image-Text Foundation Models Code
#8CuMo-7B
82.2
Accuracy· Extra Data· 2024-05-09
CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts Code
#9OFA
82
Accuracy· 2022-02-07
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Code
#10X2-VLM (large)
81.9
Accuracy· 2022-11-22
X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks Code
#11BLIP-2 ViT-G OPT 2.7B (fine-tuned)
81.74
Accuracy· 2023-01-30
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Code
#12BLIP-2 ViT-G FlanT5 XL (fine-tuned)
81.66
Accuracy· 2023-01-30
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Code
#13MMU
81.26
Accuracy· 2021-11-17
Achieving Human Parity on Visual Question Answering
#14Lyrics
81.2
Accuracy· 2023-12-08
Lyrics: Boosting Fine-grained Language-Vision Alignment and Comprehension via Semantic-aware Visual Objects
#15InternVL-C
81.2
Accuracy· 2023-12-21
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Code
#16mPLUG-2
81.11
Accuracy· 2023-02-01
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video Code
#17X2-VLM (base)
80.4
Accuracy· 2022-11-22
X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks Code
#18XFM (base)
80.4
Accuracy· 2023-01-12
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks Code
#19VAST
80.23
Accuracy· Extra Data
No paper
#20Florence
80.16
Accuracy· 2021-11-22
Florence: A New Foundation Model for Computer Vision Code
#21SimVLMSOTA
80.03
Accuracy· 2021-08-24
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Code
#22VALOR
78.46
Accuracy· Extra Data· 2023-04-17
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Code
#23Prismer
78.43
Accuracy· 2023-03-04
Prismer: A Vision-Language Model with Multi-Task Experts Code
#24X-VLM (base)
78.22
Accuracy· 2021-11-16
Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts Code
#25VK-OOD
77.9
Accuracy
No paperCode
#26Aurora (ours, r=64)
77.69
Accuracy
No paper
#27VK-OOD
76.8
Accuracy· 2023-02-11
Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis Code
#28ALBEF (14M)SOTA
75.84
Accuracy· 2021-07-16
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation Code
#29OscarSOTA
73.82
Accuracy· 2020-04-13
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks Code
#30UNITER (Large)SOTA
73.24
Accuracy· 2019-09-25
UNITER: UNiversal Image-TExt Representation Learning Code
#31X-101 grid features + MCAN
72.59
Accuracy· 2020-01-10
In Defense of Grid Features for Visual Question Answering Code
#32CFR
72.5
Accuracy· 2021-10-06
Coarse-to-Fine Reasoning for Visual Question Answering Code
#33VL-BERTLARGESOTA
71.79
Accuracy· 2019-08-22
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Code
#34ViLT-B/32
71.26
Accuracy· 2021-02-05
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision Code
#35MCAN+VC
71.21
Accuracy· 2020-02-27
Visual Commonsense R-CNN Code
#36VL-BERTBASE
71.16
Accuracy· 2019-08-22
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Code
#37VisualBERTSOTA
70.8
Accuracy· 2019-08-09
VisualBERT: A Simple and Performant Baseline for Vision and Language Code
#38LXMERT (low-magnitude pruning)
70.72
Accuracy· 2023-10-23
LXMERT Model Compression for Visual Question Answering Code
#39MCANed-6SOTA
70.63
Accuracy· 2019-06-25
Deep Modular Co-Attention Networks for Visual Question Answering Code
#40ViLBERT
70.55
Accuracy· 2019-08-06
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Code
#41BAN+Glove+CounterSOTA
70.04
Accuracy· 2018-05-21
Bilinear Attention Networks Code
#42LXMERT (Pre-train + scratch)
69.9
Accuracy· 2019-08-20
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Code
#43Image features from bottom-up attention (adaptive K, ensemble)SOTA
69.87
Accuracy· 2017-08-09
Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge Code
#44Pythia v0.3 + LoRRA
69.21
Accuracy· 2019-04-18
Towards VQA Models That Can Read Code
#45DMN
68.09
Accuracy· 2018-02-15
Learning to Count Objects in Natural Images for Visual Question Answering Code
#46LaKo
68.07
Accuracy· 2022-07-26
LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injection Code
#47MuRel
68.03
Accuracy· 2019-02-25
MUREL: Multimodal Relational Reasoning for Visual Question Answering Code
#48BLOCK
67.58
Accuracy· 2019-01-31
BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and Visual Relationship Detection Code
#49MUTANSOTA
67.42
Accuracy· 2017-05-18
MUTAN: Multimodal Tucker Fusion for Visual Question Answering Code
#50BAN2-CTI
67.4
Accuracy· 2019-09-26
Compact Trilinear Interaction for Visual Question Answering Code
#512D continuous softmax
65.96
Accuracy· 2020-06-12
Sparse and Continuous Attention Mechanisms Code
#52BLIP-2 ViT-G FlanT5 XXL (zero-shot)
65
Accuracy· 2023-01-30
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Code
#53N2NMN (ResNet-152, policy search)SOTA
64.9
Accuracy· 2017-04-18
Learning to Reason: End-to-End Module Networks for Visual Question Answering Code
#54PNP-VQA
64.8
Accuracy· 2022-10-17
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training Code
#55MCBSOTA
64.7
Accuracy· 2016-06-06
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding Code
#56RUBi
63.18
Accuracy· 2019-06-24
RUBi: Reducing Unimodal Biases in Visual Question Answering Code
#57BLIP-2 ViT-G FlanT5 XL (zero-shot)
63
Accuracy· 2023-01-30
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Code
#58BLIP-2 ViT-L FlanT5 XL (zero-shot)
62.3
Accuracy· 2023-01-30
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Code
#59Flamingo 80B
56.3
Accuracy· 2022-04-29
Flamingo: a Visual Language Model for Few-Shot Learning Code
#60LocVLM-L
56.2
Accuracy· 2024-04-11
Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs Code
#61BLIP-2 ViT-G OPT 6.7B (zero-shot)
52.6
Accuracy· 2023-01-30
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Code
#62BLIP-2 ViT-G OPT 2.7B (zero-shot)
52.3
Accuracy· 2023-01-30
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Code
#63Flamingo 9B
51.8
Accuracy· 2022-04-29
Flamingo: a Visual Language Model for Few-Shot Learning Code
#64KOSMOS-1 1.6B (zero-shot)
51
Accuracy
No paper
#65BLIP-2 ViT-L OPT 2.7B (zero-shot)
49.7
Accuracy· 2023-01-30
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Code
#66Flamingo 3B
49.2
Accuracy· 2022-04-29
Flamingo: a Visual Language Model for Few-Shot Learning Code
#67VLKD
44.5
Accuracy
No paper