Vision and Language Navigation on VLN Challenge

Metric: error (lower is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	error▲	Extra Data	Paper	Date↕	Code
1	human	1.61	No	-	-	-
2	Lily	2.5	No	-	-	-
3	Airbert	2.58	No	Airbert: In-domain Pretraining for Vision-and-La...	2021-08-20	Code
4	Global Normalization	2.99	No	-	-	-
5	Gloabl Normalization pre-explore	3	No	-	-	-
6	FOAM-Beam Search	3.05	No	-	-	-
7	Active Exploration (Beam Search)	3.07	No	-	-	-
8	VLN-Bert	3.09	No	Improving Vision-and-Language Navigation with Im...	2020-04-30	Code
9	BEVBert	3.13	No	-	-	-
10	SERL (Beam_Search)	3.21	No	-	-	-
11	Self-Supervised Auxiliary Reasoning Tasks (Beam Search)	3.24	No	Vision-Language Navigation with Self-Supervised ...	2019-11-18	-
12	CMC-AAL2	3.24	No	-	-	-
13	sponge	3.26	No	-	-	-
14	null	3.26	No	Learning to Navigate Unseen Environments: Back T...	2019-04-08	Code
15	Active Exploration (Pre-explore)	3.3	No	-	-	-
16	ADad	3.31	No	-	-	-
17	ReadNet	3.33	No	-	-	-
18	GMap	3.35	No	-	-	-
19	HOC	3.38	No	-	-	-
20	CMG-AAL	3.41	No	-	-	-
21	Lily	3.44	No	-	-	-
22	Single-run	3.52	No	-	-	-
23	explore@40 beam-search	3.55	No	-	-	-
24	DUET+PASTS	3.55	No	-	-	-
25	metaexplore	3.57	No	-	-	-
26	EnvEdit+PT	3.59	No	-	-	-
27	lxyict	3.61	No	-	-	-
28	CVPR22	3.65	No	-	-	-
29	VLN-BERT-Aug	3.67	No	-	-	-
30	TD-STP	3.68	No	-	-	-
31	Self-Supervised Auxiliary Reasoning Tasks (Pre-explore)	3.69	No	Vision-Language Navigation with Self-Supervised ...	2019-11-18	-
32	Single-Run, No Pre-Explore	3.73	No	-	-	-
33	DDL	3.74	No	-	-	-
34	Greedy, No Pre-explore	3.76	No	-	-	-
35	sliu_team	3.77	No	-	-	-
36	ESceme Single-run	3.77	No	-	-	-
37	WIN	3.78	No	-	-	-
38	WIN + RecVLN BERT	3.78	No	-	-	-
39	Single-Run	3.82	No	-	-	-
40	VLN-TreeTrans	3.85	No	-	-	-
41	Fortest	3.86	No	-	-	-
42	coefficient	3.86	No	-	-	-
43	PANDA-TingLiu	3.86	No	-	-	-
44	single-run	3.87	No	Vision-Language Navigation with Random Environme...	2021-06-15	Code
45	single-run	3.88	No	-	-	-
46	clin	3.9	No	-	-	-
47	HAMT	3.93	No	-	-	-
48	single-run	3.94	No	-	-	-
49	YBYB	3.96	No	-	-	-
50	DDL	3.97	No	-	-	-
51	Back Translation with Environmental Dropout (exploring unseen environments before testing)	3.97	No	-	-	-
52	Colab_buaa	3.98	No	-	-	-
53	hellohellohello	3.99	No	-	-	-
54	bin	4	No	-	-	-
55	Hikari	4	No	-	-	-
56	SEvol_lzy	4.02	No	-	-	-
57	GraphBert	4.02	No	-	-	-
58	Reinforced Cross-Modal Matching (optimized for SR; with beam search)	4.03	No	-	-	-
59	Geo	4.04	No	-	-	-
60	binbin	4.07	No	-	-	-
61	Single-Run, No Pre-Explore	4.09	No	-	-	-
62	reg	4.09	No	-	-	-
63	ART	4.09	No	-	-	-
64	ed	4.16	No	-	-	-
65	MARVAL	4.18	No	-	-	-
66	SIL-R2	4.21	No	-	-	-
67	GBSE	4.22	No	-	-	-
68	hellohello	4.26	No	-	-	-
69	Envdrop+SEVol+BT	4.26	No	-	-	-
70	CCC(ssm)	4.27	No	-	-	-
71	homebody	4.27	No	-	-	-
72	Tactical Rewind - long	4.29	No	Tactical Rewind: Self-Correction via Backtrackin...	2019-03-06	Code
73	single-run	4.3	No	-	-	-
74	zq	4.32	No	-	-	-
75	Single Run	4.34	No	-	-	-
76	MM2021	4.37	No	Neighbor-view Enhanced Model for Vision and Lang...	2021-07-15	Code
77	SQANv1, No Pre-explore	4.45	No	-	-	-
78	Self-Aware Co-Grounded Model	4.48	No	-	-	-
79	trysth	4.49	No	-	-	-
80	PREVALENT	4.52	No	-	-	-
81	a new baseline	4.53	No	-	-	-
82	Greedy	4.53	No	-	-	-
83	SSM	4.57	No	-	-	-
84	test-sf	4.57	No	-	-	-
85	null	4.57	No	-	-	-
86	Single-Run, No Pre-Explore	4.59	No	-	-	-
87	CMG-AAL-TCSVT	4.61	No	-	-	-
88	reward-vln	4.65	No	-	-	-
89	SEA features + AuxRN (single-run)	4.71	No	-	-	-
90	Lang-Vis-Entity VLN (Single-Run)	4.75	No	-	-	-
91	liuer	4.84	No	-	-	-
92	Speaker-Follower	4.87	No	-	-	-
93	SEA features + Env-Dropout (single-run)	4.89	No	-	-	-
94	Single-Run	4.9	No	-	-	-
95	without pre-explore, beam-search	4.95	No	-	-	-
96	DCMT	4.96	No	-	-	-
97	27k	5.11	No	-	-	-
98	Tactical Rewind - short	5.14	No	Tactical Rewind: Self-Correction via Backtrackin...	2019-03-06	Code
99	licr19	5.14	No	-	-	-
100	jmebs	5.17	No	-	-	-
101	Back Translation with Environmental Dropout (no beam search)	5.23	No	Learning to Navigate Unseen Environments: Back T...	2019-04-08	Code
102	single run	5.24	No	-	-	-
103	OAG(without pre-explore, beam-search)	5.29	No	-	-	-
104	PREVALENT	5.3	No	-	-	-
105	2m-path	5.3	No	-	-	-
106	WQ_Pretrain	5.32	No	-	-	-
107	GVLN	5.34	No	-	-	-
108	map1	5.35	No	-	-	-
109	single-run	5.37	No	-	-	-
110	SEA features + Speaker-Follower (single-run)	5.42	No	-	-	-
111	SYSU-ISE	5.45	No	-	-	-
112	Single-Run, No Pre-Explore	5.45	No	-	-	-
113	ALTR	5.49	No	Transferable Representation Learning in Vision-a...	2019-08-09	-
114	testliu	5.5	No	-	-	-
115	DoubleAttn	5.56	No	-	-	-
116	SERL	5.63	No	-	-	-
117	naive	5.64	No	-	-	-
118	Self-Monitoring Navigation Agent (no beam search; Progress Inference)	5.67	No	Self-Monitoring Navigation Agent via Auxiliary P...	2019-01-10	Code
119	The Regretful Agent (no beam search; greedy action selection)	5.69	No	The Regretful Agent: Heuristic-Aided Navigation ...	2019-03-05	Code
120	SERL (no_augmented)	5.7	No	-	-	-
121	selfmoni0	5.8	No	-	-	-
122	speaker_follower_tesk2	6.01	No	-	-	-
123	Environment-Agnostic Multitask Learning	6.03	No	Environment-agnostic Multitask Learning for Natu...	2020-03-01	Code
124	Reinforced Cross-Modal Matching (single trajectory; NO beam search)	6.12	No	-	-	-
125	PTA	6.17	No	-	-	-
126	Khanh Nguyen	6.46	No	-	-	-
127	Dynamic Convolutional Filters	6.55	No	-	-	-
128	HAIL	6.61	No	-	-	-
129	rcm_test	6.67	No	-	-	-
130	base_1	6.68	No	-	-	-
131	AnonymousTeam	6.89	No	-	-	-
132	ai like samurai with PNasNet5Large	6.95	No	-	-	-
133	fuse_1	7.03	No	-	-	-
134	base_0	7.2	No	-	-	-
135	test_an	7.47	No	-	-	-
136	Look Before You Leap	7.53	No	-	-	-
137	no	7.63	No	-	-	-
138	Seq2Seq Baseline	7.85	No	-	-	-
139	zhangyong	7.99	No	-	-	-
140	X-Modal	8.56	No	-	-	-
141	zy123	8.82	No	-	-	-
142	Random Agent	9.79	No	-	-	-
143	zzzzzzzz55768	9.8	No	-	-	-
144	15458	9.93	No	-	-	-
145	1111	12.29	No	-	-	-

#1human
1.61
error
No paper
#2Lily
2.5
error
No paper
#3AirbertSOTA
2.58
error· 2021-08-20
Airbert: In-domain Pretraining for Vision-and-Language Navigation Code
#4Global Normalization
2.99
error
No paper
#5Gloabl Normalization pre-explore
3
error
No paper
#6FOAM-Beam Search
3.05
error
No paper
#7Active Exploration (Beam Search)
3.07
error
No paper
#8VLN-BertSOTA
3.09
error· 2020-04-30
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web Code
#9BEVBert
3.13
error
No paper
#10SERL (Beam_Search)
3.21
error
No paper
#11Self-Supervised Auxiliary Reasoning Tasks (Beam Search)SOTA
3.24
error· 2019-11-18
Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
#12CMC-AAL2
3.24
error
No paper
#13sponge
3.26
error
No paper
#14nullSOTA
3.26
error· 2019-04-08
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout Code
#15Active Exploration (Pre-explore)
3.3
error
No paper
#16ADad
3.31
error
No paper
#17ReadNet
3.33
error
No paper
#18GMap
3.35
error
No paper
#19HOC
3.38
error
No paper
#20CMG-AAL
3.41
error
No paper
#21Lily
3.44
error
No paper
#22Single-run
3.52
error
No paper
#23explore@40 beam-search
3.55
error
No paper
#24DUET+PASTS
3.55
error
No paper
#25metaexplore
3.57
error
No paper
#26EnvEdit+PT
3.59
error
No paper
#27lxyict
3.61
error
No paper
#28CVPR22
3.65
error
No paper
#29VLN-BERT-Aug
3.67
error
No paper
#30TD-STP
3.68
error
No paper
#31Self-Supervised Auxiliary Reasoning Tasks (Pre-explore)
3.69
error· 2019-11-18
Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
#32Single-Run, No Pre-Explore
3.73
error
No paper
#33DDL
3.74
error
No paper
#34Greedy, No Pre-explore
3.76
error
No paper
#35sliu_team
3.77
error
No paper
#36ESceme Single-run
3.77
error
No paper
#37WIN
3.78
error
No paper
#38WIN + RecVLN BERT
3.78
error
No paper
#39Single-Run
3.82
error
No paper
#40VLN-TreeTrans
3.85
error
No paper
#41Fortest
3.86
error
No paper
#42coefficient
3.86
error
No paper
#43PANDA-TingLiu
3.86
error
No paper
#44single-run
3.87
error· 2021-06-15
Vision-Language Navigation with Random Environmental Mixup Code
#45single-run
3.88
error
No paper
#46clin
3.9
error
No paper
#47HAMT
3.93
error
No paper
#48single-run
3.94
error
No paper
#49YBYB
3.96
error
No paper
#50DDL
3.97
error
No paper
#51Back Translation with Environmental Dropout (exploring unseen environments before testing)
3.97
error
No paper
#52Colab_buaa
3.98
error
No paper
#53hellohellohello
3.99
error
No paper
#54bin
4
error
No paper
#55Hikari
4
error
No paper
#56SEvol_lzy
4.02
error
No paper
#57GraphBert
4.02
error
No paper
#58Reinforced Cross-Modal Matching (optimized for SR; with beam search)
4.03
error
No paper
#59Geo
4.04
error
No paper
#60binbin
4.07
error
No paper
#61Single-Run, No Pre-Explore
4.09
error
No paper
#62reg
4.09
error
No paper
#63ART
4.09
error
No paper
#64ed
4.16
error
No paper
#65MARVAL
4.18
error
No paper
#66SIL-R2
4.21
error
No paper
#67GBSE
4.22
error
No paper
#68hellohello
4.26
error
No paper
#69Envdrop+SEVol+BT
4.26
error
No paper
#70CCC(ssm)
4.27
error
No paper
#71homebody
4.27
error
No paper
#72Tactical Rewind - longSOTA
4.29
error· 2019-03-06
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation Code
#73single-run
4.3
error
No paper
#74zq
4.32
error
No paper
#75Single Run
4.34
error
No paper
#76MM2021
4.37
error· 2021-07-15
Neighbor-view Enhanced Model for Vision and Language Navigation Code
#77SQANv1, No Pre-explore
4.45
error
No paper
#78Self-Aware Co-Grounded Model
4.48
error
No paper
#79trysth
4.49
error
No paper
#80PREVALENT
4.52
error
No paper
#81a new baseline
4.53
error
No paper
#82Greedy
4.53
error
No paper
#83SSM
4.57
error
No paper
#84test-sf
4.57
error
No paper
#85null
4.57
error
No paper
#86Single-Run, No Pre-Explore
4.59
error
No paper
#87CMG-AAL-TCSVT
4.61
error
No paper
#88reward-vln
4.65
error
No paper
#89SEA features + AuxRN (single-run)
4.71
error
No paper
#90Lang-Vis-Entity VLN (Single-Run)
4.75
error
No paper
#91liuer
4.84
error
No paper
#92Speaker-Follower
4.87
error
No paper
#93SEA features + Env-Dropout (single-run)
4.89
error
No paper
#94Single-Run
4.9
error
No paper
#95without pre-explore, beam-search
4.95
error
No paper
#96DCMT
4.96
error
No paper
#9727k
5.11
error
No paper
#98Tactical Rewind - shortSOTA
5.14
error· 2019-03-06
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation Code
#99licr19
5.14
error
No paper
#100jmebs
5.17
error
No paper
#101Back Translation with Environmental Dropout (no beam search)
5.23
error· 2019-04-08
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout Code
#102single run
5.24
error
No paper
#103OAG(without pre-explore, beam-search)
5.29
error
No paper
#104PREVALENT
5.3
error
No paper
#1052m-path
5.3
error
No paper
#106WQ_Pretrain
5.32
error
No paper
#107GVLN
5.34
error
No paper
#108map1
5.35
error
No paper
#109single-run
5.37
error
No paper
#110SEA features + Speaker-Follower (single-run)
5.42
error
No paper
#111SYSU-ISE
5.45
error
No paper
#112Single-Run, No Pre-Explore
5.45
error
No paper
#113ALTR
5.49
error· 2019-08-09
Transferable Representation Learning in Vision-and-Language Navigation
#114testliu
5.5
error
No paper
#115DoubleAttn
5.56
error
No paper
#116SERL
5.63
error
No paper
#117naive
5.64
error
No paper
#118Self-Monitoring Navigation Agent (no beam search; Progress Inference)SOTA
5.67
error· 2019-01-10
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation Code
#119The Regretful Agent (no beam search; greedy action selection)
5.69
error· 2019-03-05
The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation Code
#120SERL (no_augmented)
5.7
error
No paper
#121selfmoni0
5.8
error
No paper
#122speaker_follower_tesk2
6.01
error
No paper
#123Environment-Agnostic Multitask Learning
6.03
error· 2020-03-01
Environment-agnostic Multitask Learning for Natural Language Grounded Navigation Code
#124Reinforced Cross-Modal Matching (single trajectory; NO beam search)
6.12
error
No paper
#125PTA
6.17
error
No paper
#126Khanh Nguyen
6.46
error
No paper
#127Dynamic Convolutional Filters
6.55
error
No paper
#128HAIL
6.61
error
No paper
#129rcm_test
6.67
error
No paper
#130base_1
6.68
error
No paper
#131AnonymousTeam
6.89
error
No paper
#132ai like samurai with PNasNet5Large
6.95
error
No paper
#133fuse_1
7.03
error
No paper
#134base_0
7.2
error
No paper
#135test_an
7.47
error
No paper
#136Look Before You Leap
7.53
error
No paper
#137no
7.63
error
No paper
#138Seq2Seq Baseline
7.85
error
No paper
#139zhangyong
7.99
error
No paper
#140X-Modal
8.56
error
No paper
#141zy123
8.82
error
No paper
#142Random Agent
9.79
error
No paper
#143zzzzzzzz55768
9.8
error
No paper
#14415458
9.93
error
No paper
#1451111
12.29
error
No paper