Visual Question Answering (VQA) on GQA Test2019

Metric: Consistency (higher is better)

LeaderboardDataset

Loading chart...

Results

#	Model↕	Consistency▼	Extra Data	Paper	Date↕	Code
1	human	98.4	No	-	-	-
2	VinVL-DPT	94.37	No	-	-	-
3	1-gqa	94.36	No	-	-	-
4	Single Model	94.35	No	VinVL: Revisiting Visual Representations in Visi...	2021-01-02	Code
5	VinVL+L	94	No	-	-	Code
6	MDETR	93.95	No	-	-	-
7	NSM ensemble (updated)	93.83	No	-	-	-
8	Musan	93.72	No	-	-	-
9	NSM single (updated)	93.25	No	-	-	-
10	LXR955, Ensemble	93.1	No	LXMERT: Learning Cross-Modality Encoder Represen...	2019-08-20	Code
11	IQA (single)	93.02	No	-	-	-
12	LININ	92.66	No	-	-	-
13	Ensemble10	92.61	No	-	-	-
14	Meta Module, Single	92.49	No	-	-	-
15	MIL-nbgao	91.76	No	-	-	-
16	Feb_ft2_mergeadd_weightalllstm_picklocw_box5_prep	91.71	No	-	-	-
17	BAN	91.7	No	-	-	-
18	Wayne	91.54	No	-	-	-
19	DREAM+Unicoder-VL (MSRA)	91.47	No	-	-	-
20	PVR	91.43	No	-	-	-
21	lxmert-adv-txt	91.13	No	-	-	-
22	lxmert-adv-txt	91.08	No	-	-	-
23	Coarse-to-Fine Reasoning, Single Model	90.96	No	-	-	-
24	happyTeam	90.96	No	-	-	-
25	fisher	90.77	No	-	-	-
26	wcf-fight	90.49	No	-	-	-
27	GRN	90.31	No	Bilinear Graph Networks for Visual Question Answ...	2019-07-23	-
28	45	90.3	No	-	-	-
29	Partial-MSP	90.29	No	-	-	-
30	full_nsp_ft_results_submit_predict.json	90.16	No	-	-	-
31	full_nsp_mlm_ft_joint_results_submit_predict.json	90.05	No	-	-	-
32	Single	89.84	No	-	-	-
33	fbe20v3.json	89.81	No	-	-	-
34	QGCRGN	89.81	No	-	-	-
35	mlmbert	89.77	No	-	-	-
36	bert_v1	89.77	No	-	-	-
37	test gqa	89.77	No	-	-	-
38	vv69	89.69	No	-	-	-
39	xpj	89.65	No	-	-	-
40	TESTOVQA007	89.65	No	-	-	-
41	Future_Test_team	89.61	No	-	-	-
42	LXR955, Single Model	89.59	No	LXMERT: Learning Cross-Modality Encoder Represen...	2019-08-20	Code
43	tmp	89.58	No	-	-	-
44	ours-4-gqa_el_tag_v4__pretrain_rel_tag_dist_tc_v7_checkpoint-47-157510-best-4.json	89.58	No	-	-	-
45	gaochongyang9	89.52	No	-	-	-
46	gbert1	89.52	No	-	-	-
47	IIE_Morningstar	89.47	No	-	-	-
48	DAM	89.43	No	-	-	-
49	Inspur	89.32	No	-	-	-
50	DL16	89.21	No	-	-	-
51	LW	89.18	No	-	-	-
52	SSRP	89.17	No	-	-	-
53	BgTest	89.14	No	-	-	-
54	ckpt 19 exp 90	89.08	No	-	-	-
55	mcmi	89.05	No	-	-	-
56	TRRNet (Ensemble)	89	No	-	-	-
57	UNITER + MAC + Graph Networks	88.94	No	-	-	-
58	MSM@MSRA	88.92	No	-	-	-
59	LXMERT-S	88.9	No	-	-	-
60	UCM	88.68	No	-	-	-
61	rishabh_test	88.63	No	-	-	-
62	VqaStar-UCAS-SARI	88.3	No	-	-	-
63	UCAS-SARI	88.25	No	-	-	-
64	prompt IMT-16	88.2	No	-	-	-
65	stu09e	88.01	No	-	-	-
66	RSN (Single Model)	87.74	No	-	-	-
67	total14	87.71	No	-	-	-
68	rsa-14word	87.61	No	-	-	-
69	result_run_2647872_epoch11	87.6	No	-	-	-
70	5TMT-qe+o	87.47	No	-	-	-
71	DeeTee	87.36	No	-	-	-
72	MLVQA (single)	87.35	No	-	-	-
73	ST_VQA	87.16	No	-	-	-
74	270	86.94	No	-	-	-
75	MMT-VQA	86.86	No	-	-	-
76	REX	86.85	No	-	-	-
77	HDU_ZWF	86.1	No	-	-	-
78	LOGNet+VLR	85.51	No	-	-	-
79	IWantADonut	85.46	No	-	-	-
80	graphRepresentation， Single	85.45	No	-	-	-
81	GM6_9_2_train	85.12	No	-	-	-
82	Improved SNMN	85.11	No	-	-	-
83	GIN	84.99	No	-	-	-
84	F205	84.7	No	-	-	-
85	LCGN	84.68	No	-	-	-
86	glimple_all	84.64	No	-	-	-
87	UJCNN	84.36	No	-	-	-
88	Testify	84.35	No	-	-	-
89	abc_test	84.17	No	-	-	-
90	LAS	84.16	No	-	-	-
91	mfb+bert	83.85	No	-	-	-
92	RAM_BUGGY	83.68	No	-	-	-
93	Deepblue_Semantics	83.52	No	-	-	-
94	KU	83.47	No	-	-	-
95	RSN (Single Model)_v6	83.32	No	-	-	-
96	RD	83.2	No	-	-	-
97	nogg	83.06	No	-	-	-
98	Eden_test	82.71	No	-	-	-
99	Space Cat	82.44	No	-	-	-
100	vips	82.36	No	-	-	-
101	MReaL	82.21	No	-	-	-
102	MAC	81.59	No	-	-	-
103	RES	81.19	No	-	-	-
104	test	81.14	No	-	-	-
105	mac_qin	80.44	No	-	-	-
106	Sorbonne	80.33	No	-	-	-
107	ZhaoLab	80.2	No	-	-	-
108	Mithrandir	80.2	No	-	-	-
109	happy	79.7	No	-	-	-
110	BottomUp	78.71	No	Bottom-Up and Top-Down Attention for Image Capti...	2017-07-25	Code
111	sparsemax15	78.51	No	-	-	-
112	MJ	78.34	No	-	-	-
113	Kakao Brain	77.02	No	-	-	-
114	LSTM+CNN	74.57	No	-	-	-
115	Academia Sinica	74.11	No	-	-	-
116	test	73.71	No	-	-	-
117	113	70.57	No	-	-	-
118	Mycsulb	69.93	No	-	-	-
119	LSTM	68.68	No	-	-	-
120	bear	67.71	No	-	-	-
121	Fj	63.96	No	-	-	-
122	CHAIR	63.51	No	-	-	-
123	CNN	62.4	No	-	-	-
124	muc_ai	55.23	No	-	-	-
125	LocalPrior	54.04	No	-	-	-
126	GlobalPrior	51.69	No	-	-	-
127	Ediburgh-Mila-UCLA	38.18	No	-	-	-

Model↕

Consistency▼

Extra Data

Paper

Date↕

Code

human

98.4

VinVL-DPT

94.37

1-gqa

94.36

Single Model

94.35

VinVL: Revisiting Visual Representations in Visi...

2021-01-02

Code

VinVL+L

Code

MDETR

93.95

NSM ensemble (updated)

93.83

Musan

93.72

NSM single (updated)

93.25

LXR955, Ensemble

93.1

LXMERT: Learning Cross-Modality Encoder Represen...

2019-08-20

Code

IQA (single)

93.02

LININ

92.66

Ensemble10

92.61

Meta Module, Single

92.49

MIL-nbgao

91.76

Feb_ft2_mergeadd_weightalllstm_picklocw_box5_prep

91.71

BAN

91.7

Wayne

91.54

DREAM+Unicoder-VL (MSRA)

91.47

PVR

91.43

lxmert-adv-txt

91.13

lxmert-adv-txt

91.08

Coarse-to-Fine Reasoning, Single Model

90.96

happyTeam

90.96

fisher

90.77

wcf-fight

90.49

GRN

90.31

Bilinear Graph Networks for Visual Question Answ...

2019-07-23

90.3

Partial-MSP

90.29

full_nsp_ft_results_submit_predict.json

90.16

full_nsp_mlm_ft_joint_results_submit_predict.json

90.05

Single

89.84

fbe20v3.json

89.81

QGCRGN

89.81

mlmbert

89.77

bert_v1

89.77

test gqa

89.77

vv69

89.69

xpj

89.65

TESTOVQA007

89.65

Future_Test_team

89.61

LXR955, Single Model

89.59

LXMERT: Learning Cross-Modality Encoder Represen...

2019-08-20

Code

tmp

89.58

ours-4-gqa_el_tag_v4__pretrain_rel_tag_dist_tc_v7_checkpoint-47-157510-best-4.json

89.58

gaochongyang9

89.52

gbert1

89.52

IIE_Morningstar

89.47

DAM

89.43

Inspur

89.32

DL16

89.21

89.18

SSRP

89.17

BgTest

89.14

ckpt 19 exp 90

89.08

mcmi

89.05

TRRNet (Ensemble)

UNITER + MAC + Graph Networks

88.94

MSM@MSRA

88.92

LXMERT-S

88.9

UCM

88.68

rishabh_test

88.63

VqaStar-UCAS-SARI

88.3

UCAS-SARI

88.25

prompt IMT-16

88.2

stu09e

88.01

RSN (Single Model)

87.74

total14

87.71

rsa-14word

87.61

result_run_2647872_epoch11

87.6

5TMT-qe+o

87.47

DeeTee

87.36

MLVQA (single)

87.35

ST_VQA

87.16

270

86.94

MMT-VQA

86.86

REX

86.85

HDU_ZWF

86.1

LOGNet+VLR

85.51

IWantADonut

85.46

graphRepresentation， Single

85.45

GM6_9_2_train

85.12

Improved SNMN

85.11

GIN

84.99

F205

84.7

LCGN

84.68

glimple_all

84.64

UJCNN

84.36

Testify

84.35

abc_test

84.17

LAS

84.16

mfb+bert

83.85

RAM_BUGGY

83.68

Deepblue_Semantics

83.52

83.47

RSN (Single Model)_v6

83.32

83.2

nogg

83.06

Eden_test

82.71

Space Cat

82.44

100

vips

82.36

101

MReaL

82.21

102

MAC

81.59

103

RES

81.19

104

test

81.14

105

mac_qin

80.44

106

Sorbonne

80.33

107

ZhaoLab

80.2

108

Mithrandir

80.2

109

happy

79.7

110

BottomUp

78.71

Bottom-Up and Top-Down Attention for Image Capti...

2017-07-25

Code

111

sparsemax15

78.51

112

78.34

113

Kakao Brain

77.02

114

LSTM+CNN

74.57

115

Academia Sinica

74.11

116

test

73.71

117

113

70.57

118

Mycsulb

69.93

119

LSTM

68.68

120

bear

67.71

121

63.96

122

CHAIR

63.51

123

CNN

62.4

124

muc_ai

55.23

125

LocalPrior

54.04

126

GlobalPrior

51.69

127

Ediburgh-Mila-UCLA

38.18