Vision and Language Navigation on VLN Challenge

Metric: oracle success (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	oracle success▼	Extra Data	Paper	Date↕	Code
1	FOAM-Beam Search	1	No	-	-	-
2	Lily	0.99	No	-	-	-
3	Airbert	0.99	No	Airbert: In-domain Pretraining for Vision-and-La...	2021-08-20	Code
4	Global Normalization	0.99	No	-	-	-
5	explore@40 beam-search	0.99	No	-	-	-
6	VLN-Bert	0.99	No	Improving Vision-and-Language Navigation with Im...	2020-04-30	Code
7	SERL (Beam_Search)	0.99	No	-	-	-
8	ADad	0.99	No	-	-	-
9	null	0.99	No	Learning to Navigate Unseen Environments: Back T...	2019-04-08	Code
10	Self-Aware Co-Grounded Model	0.97	No	-	-	-
11	Reinforced Cross-Modal Matching (optimized for SR; with beam search)	0.96	No	-	-	-
12	null	0.96	No	-	-	-
13	Speaker-Follower	0.96	No	-	-	-
14	Active Exploration (Beam Search)	0.94	No	-	-	-
15	human	0.9	No	-	-	-
16	Tactical Rewind - long	0.9	No	Tactical Rewind: Self-Correction via Backtrackin...	2019-03-06	Code
17	BEVBert	0.81	No	-	-	-
18	Self-Supervised Auxiliary Reasoning Tasks (Beam Search)	0.81	No	Vision-Language Navigation with Self-Supervised ...	2019-11-18	-
19	GMap	0.8	No	-	-	-
20	Gloabl Normalization pre-explore	0.8	No	-	-	-
21	Lily	0.79	No	-	-	-
22	HOC	0.79	No	-	-	-
23	ReadNet	0.78	No	-	-	-
24	metaexplore	0.77	No	-	-	-
25	sponge	0.77	No	-	-	-
26	lxyict	0.77	No	-	-	-
27	Active Exploration (Pre-explore)	0.77	No	-	-	-
28	DUET+PASTS	0.76	No	-	-	-
29	CVPR22	0.76	No	-	-	-
30	CMC-AAL2	0.76	No	-	-	-
31	CMG-AAL	0.76	No	-	-	-
32	Single-run	0.75	No	-	-	-
33	Self-Supervised Auxiliary Reasoning Tasks (Pre-explore)	0.75	No	Vision-Language Navigation with Self-Supervised ...	2019-11-18	-
34	VLN-TreeTrans	0.74	No	-	-	-
35	EnvEdit+PT	0.73	No	-	-	-
36	DDL	0.73	No	-	-	-
37	Single-Run, No Pre-Explore	0.73	No	-	-	-
38	VLN-BERT-Aug	0.73	No	-	-	-
39	TD-STP	0.72	No	-	-	-
40	Fortest	0.72	No	-	-	-
41	ESceme Single-run	0.72	No	-	-	-
42	single-run	0.72	No	Vision-Language Navigation with Random Environme...	2021-06-15	Code
43	Single-Run	0.72	No	-	-	-
44	HAMT	0.72	No	-	-	-
45	CCC(ssm)	0.72	No	-	-	-
46	GraphBert	0.72	No	-	-	-
47	sliu_team	0.71	No	-	-	-
48	WIN	0.71	No	-	-	-
49	WIN + RecVLN BERT	0.71	No	-	-	-
50	coefficient	0.71	No	-	-	-
51	Greedy, No Pre-explore	0.71	No	-	-	-
52	DDL	0.71	No	-	-	-
53	single-run	0.71	No	-	-	-
54	PANDA-TingLiu	0.71	No	-	-	-
55	single-run	0.71	No	-	-	-
56	SEvol_lzy	0.71	No	-	-	-
57	hellohellohello	0.71	No	-	-	-
58	Single Run	0.71	No	-	-	-
59	bin	0.7	No	-	-	-
60	clin	0.7	No	-	-	-
61	Back Translation with Environmental Dropout (exploring unseen environments before testing)	0.7	No	-	-	-
62	Colab_buaa	0.7	No	-	-	-
63	Geo	0.7	No	-	-	-
64	YBYB	0.7	No	-	-	-
65	Single-Run, No Pre-Explore	0.7	No	-	-	-
66	reg	0.7	No	-	-	-
67	ART	0.7	No	-	-	-
68	binbin	0.7	No	-	-	-
69	SSM	0.7	No	-	-	-
70	Hikari	0.69	No	-	-	-
71	ed	0.69	No	-	-	-
72	zq	0.69	No	-	-	-
73	GBSE	0.69	No	-	-	-
74	liuer	0.69	No	-	-	-
75	single-run	0.68	No	-	-	-
76	MARVAL	0.67	No	-	-	-
77	hellohello	0.67	No	-	-	-
78	homebody	0.67	No	-	-	-
79	SIL-R2	0.67	No	-	-	-
80	trysth	0.67	No	-	-	-
81	SQANv1, No Pre-explore	0.67	No	-	-	-
82	Envdrop+SEVol+BT	0.66	No	-	-	-
83	a new baseline	0.66	No	-	-	-
84	MM2021	0.66	No	Neighbor-view Enhanced Model for Vision and Lang...	2021-07-15	Code
85	without pre-explore, beam-search	0.65	No	-	-	-
86	CMG-AAL-TCSVT	0.65	No	-	-	-
87	Single-Run, No Pre-Explore	0.65	No	-	-	-
88	test-sf	0.65	No	-	-	-
89	OAG(without pre-explore, beam-search)	0.65	No	-	-	-
90	SEA features + AuxRN (single-run)	0.64	No	-	-	-
91	PREVALENT	0.64	No	-	-	-
92	Tactical Rewind - short	0.64	No	Tactical Rewind: Self-Correction via Backtrackin...	2019-03-06	Code
93	single run	0.64	No	-	-	-
94	map1	0.64	No	-	-	-
95	Greedy	0.63	No	-	-	-
96	jmebs	0.63	No	-	-	-
97	SEA features + Env-Dropout (single-run)	0.63	No	-	-	-
98	WQ_Pretrain	0.63	No	-	-	-
99	reward-vln	0.62	No	-	-	-
100	Single-Run	0.62	No	-	-	-
101	27k	0.62	No	-	-	-
102	Lang-Vis-Entity VLN (Single-Run)	0.61	No	-	-	-
103	DCMT	0.61	No	-	-	-
104	PREVALENT	0.61	No	-	-	-
105	GVLN	0.61	No	-	-	-
106	SERL	0.61	No	-	-	-
107	2m-path	0.61	No	-	-	-
108	no	0.61	No	-	-	-
109	SYSU-ISE	0.6	No	-	-	-
110	licr19	0.6	No	-	-	-
111	single-run	0.59	No	Vision-Language Navigation with Random Environme...	2021-06-15	Code
112	Back Translation with Environmental Dropout (no beam search)	0.59	No	Learning to Navigate Unseen Environments: Back T...	2019-04-08	Code
113	Self-Monitoring Navigation Agent (no beam search; Progress Inference)	0.59	No	Self-Monitoring Navigation Agent via Auxiliary P...	2019-01-10	Code
114	SERL (no_augmented)	0.57	No	-	-	-
115	Single-Run, No Pre-Explore	0.57	No	-	-	-
116	testliu	0.57	No	-	-	-
117	selfmoni0	0.57	No	-	-	-
118	DoubleAttn	0.57	No	-	-	-
119	The Regretful Agent (no beam search; greedy action selection)	0.56	No	The Regretful Agent: Heuristic-Aided Navigation ...	2019-03-05	Code
120	ALTR	0.56	No	Transferable Representation Learning in Vision-a...	2019-08-09	-
121	Environment-Agnostic Multitask Learning	0.56	No	Environment-agnostic Multitask Learning for Natu...	2020-03-01	Code
122	SEA features + Speaker-Follower (single-run)	0.55	No	-	-	-
123	naive	0.53	No	-	-	-
124	speaker_follower_tesk2	0.53	No	-	-	-
125	Reinforced Cross-Modal Matching (single trajectory; NO beam search)	0.5	No	-	-	-
126	1111	0.49	No	-	-	-
127	PTA	0.47	No	-	-	-
128	HAIL	0.46	No	-	-	-
129	Khanh Nguyen	0.45	No	-	-	-
130	Dynamic Convolutional Filters	0.45	No	-	-	-
131	rcm_test	0.44	No	-	-	-
132	ai like samurai with PNasNet5Large	0.43	No	-	-	-
133	base_1	0.43	No	-	-	-
134	AnonymousTeam	0.41	No	-	-	-
135	fuse_1	0.4	No	-	-	-
136	base_0	0.4	No	-	-	-
137	test_an	0.34	No	-	-	-
138	Look Before You Leap	0.32	No	-	-	-
139	X-Modal	0.32	No	-	-	-
140	Seq2Seq Baseline	0.27	No	-	-	-
141	zhangyong	0.26	No	-	-	-
142	zy123	0.24	No	-	-	-
143	zzzzzzzz55768	0.19	No	-	-	-
144	Random Agent	0.18	No	-	-	-
145	15458	0	No	-	-	-

#1FOAM-Beam Search
1
oracle success
No paper
#2Lily
0.99
oracle success
No paper
#3Airbert
0.99
oracle success· 2021-08-20
Airbert: In-domain Pretraining for Vision-and-Language Navigation Code
#4Global Normalization
0.99
oracle success
No paper
#5explore@40 beam-search
0.99
oracle success
No paper
#6VLN-Bert
0.99
oracle success· 2020-04-30
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web Code
#7SERL (Beam_Search)
0.99
oracle success
No paper
#8ADad
0.99
oracle success
No paper
#9nullSOTA
0.99
oracle success· 2019-04-08
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout Code
#10Self-Aware Co-Grounded Model
0.97
oracle success
No paper
#11Reinforced Cross-Modal Matching (optimized for SR; with beam search)
0.96
oracle success
No paper
#12null
0.96
oracle success
No paper
#13Speaker-Follower
0.96
oracle success
No paper
#14Active Exploration (Beam Search)
0.94
oracle success
No paper
#15human
0.9
oracle success
No paper
#16Tactical Rewind - longSOTA
0.9
oracle success· 2019-03-06
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation Code
#17BEVBert
0.81
oracle success
No paper
#18Self-Supervised Auxiliary Reasoning Tasks (Beam Search)
0.81
oracle success· 2019-11-18
Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
#19GMap
0.8
oracle success
No paper
#20Gloabl Normalization pre-explore
0.8
oracle success
No paper
#21Lily
0.79
oracle success
No paper
#22HOC
0.79
oracle success
No paper
#23ReadNet
0.78
oracle success
No paper
#24metaexplore
0.77
oracle success
No paper
#25sponge
0.77
oracle success
No paper
#26lxyict
0.77
oracle success
No paper
#27Active Exploration (Pre-explore)
0.77
oracle success
No paper
#28DUET+PASTS
0.76
oracle success
No paper
#29CVPR22
0.76
oracle success
No paper
#30CMC-AAL2
0.76
oracle success
No paper
#31CMG-AAL
0.76
oracle success
No paper
#32Single-run
0.75
oracle success
No paper
#33Self-Supervised Auxiliary Reasoning Tasks (Pre-explore)
0.75
oracle success· 2019-11-18
Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
#34VLN-TreeTrans
0.74
oracle success
No paper
#35EnvEdit+PT
0.73
oracle success
No paper
#36DDL
0.73
oracle success
No paper
#37Single-Run, No Pre-Explore
0.73
oracle success
No paper
#38VLN-BERT-Aug
0.73
oracle success
No paper
#39TD-STP
0.72
oracle success
No paper
#40Fortest
0.72
oracle success
No paper
#41ESceme Single-run
0.72
oracle success
No paper
#42single-run
0.72
oracle success· 2021-06-15
Vision-Language Navigation with Random Environmental Mixup Code
#43Single-Run
0.72
oracle success
No paper
#44HAMT
0.72
oracle success
No paper
#45CCC(ssm)
0.72
oracle success
No paper
#46GraphBert
0.72
oracle success
No paper
#47sliu_team
0.71
oracle success
No paper
#48WIN
0.71
oracle success
No paper
#49WIN + RecVLN BERT
0.71
oracle success
No paper
#50coefficient
0.71
oracle success
No paper
#51Greedy, No Pre-explore
0.71
oracle success
No paper
#52DDL
0.71
oracle success
No paper
#53single-run
0.71
oracle success
No paper
#54PANDA-TingLiu
0.71
oracle success
No paper
#55single-run
0.71
oracle success
No paper
#56SEvol_lzy
0.71
oracle success
No paper
#57hellohellohello
0.71
oracle success
No paper
#58Single Run
0.71
oracle success
No paper
#59bin
0.7
oracle success
No paper
#60clin
0.7
oracle success
No paper
#61Back Translation with Environmental Dropout (exploring unseen environments before testing)
0.7
oracle success
No paper
#62Colab_buaa
0.7
oracle success
No paper
#63Geo
0.7
oracle success
No paper
#64YBYB
0.7
oracle success
No paper
#65Single-Run, No Pre-Explore
0.7
oracle success
No paper
#66reg
0.7
oracle success
No paper
#67ART
0.7
oracle success
No paper
#68binbin
0.7
oracle success
No paper
#69SSM
0.7
oracle success
No paper
#70Hikari
0.69
oracle success
No paper
#71ed
0.69
oracle success
No paper
#72zq
0.69
oracle success
No paper
#73GBSE
0.69
oracle success
No paper
#74liuer
0.69
oracle success
No paper
#75single-run
0.68
oracle success
No paper
#76MARVAL
0.67
oracle success
No paper
#77hellohello
0.67
oracle success
No paper
#78homebody
0.67
oracle success
No paper
#79SIL-R2
0.67
oracle success
No paper
#80trysth
0.67
oracle success
No paper
#81SQANv1, No Pre-explore
0.67
oracle success
No paper
#82Envdrop+SEVol+BT
0.66
oracle success
No paper
#83a new baseline
0.66
oracle success
No paper
#84MM2021
0.66
oracle success· 2021-07-15
Neighbor-view Enhanced Model for Vision and Language Navigation Code
#85without pre-explore, beam-search
0.65
oracle success
No paper
#86CMG-AAL-TCSVT
0.65
oracle success
No paper
#87Single-Run, No Pre-Explore
0.65
oracle success
No paper
#88test-sf
0.65
oracle success
No paper
#89OAG(without pre-explore, beam-search)
0.65
oracle success
No paper
#90SEA features + AuxRN (single-run)
0.64
oracle success
No paper
#91PREVALENT
0.64
oracle success
No paper
#92Tactical Rewind - short
0.64
oracle success· 2019-03-06
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation Code
#93single run
0.64
oracle success
No paper
#94map1
0.64
oracle success
No paper
#95Greedy
0.63
oracle success
No paper
#96jmebs
0.63
oracle success
No paper
#97SEA features + Env-Dropout (single-run)
0.63
oracle success
No paper
#98WQ_Pretrain
0.63
oracle success
No paper
#99reward-vln
0.62
oracle success
No paper
#100Single-Run
0.62
oracle success
No paper
#10127k
0.62
oracle success
No paper
#102Lang-Vis-Entity VLN (Single-Run)
0.61
oracle success
No paper
#103DCMT
0.61
oracle success
No paper
#104PREVALENT
0.61
oracle success
No paper
#105GVLN
0.61
oracle success
No paper
#106SERL
0.61
oracle success
No paper
#1072m-path
0.61
oracle success
No paper
#108no
0.61
oracle success
No paper
#109SYSU-ISE
0.6
oracle success
No paper
#110licr19
0.6
oracle success
No paper
#111single-run
0.59
oracle success· 2021-06-15
Vision-Language Navigation with Random Environmental Mixup Code
#112Back Translation with Environmental Dropout (no beam search)
0.59
oracle success· 2019-04-08
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout Code
#113Self-Monitoring Navigation Agent (no beam search; Progress Inference)SOTA
0.59
oracle success· 2019-01-10
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation Code
#114SERL (no_augmented)
0.57
oracle success
No paper
#115Single-Run, No Pre-Explore
0.57
oracle success
No paper
#116testliu
0.57
oracle success
No paper
#117selfmoni0
0.57
oracle success
No paper
#118DoubleAttn
0.57
oracle success
No paper
#119The Regretful Agent (no beam search; greedy action selection)
0.56
oracle success· 2019-03-05
The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation Code
#120ALTR
0.56
oracle success· 2019-08-09
Transferable Representation Learning in Vision-and-Language Navigation
#121Environment-Agnostic Multitask Learning
0.56
oracle success· 2020-03-01
Environment-agnostic Multitask Learning for Natural Language Grounded Navigation Code
#122SEA features + Speaker-Follower (single-run)
0.55
oracle success
No paper
#123naive
0.53
oracle success
No paper
#124speaker_follower_tesk2
0.53
oracle success
No paper
#125Reinforced Cross-Modal Matching (single trajectory; NO beam search)
0.5
oracle success
No paper
#1261111
0.49
oracle success
No paper
#127PTA
0.47
oracle success
No paper
#128HAIL
0.46
oracle success
No paper
#129Khanh Nguyen
0.45
oracle success
No paper
#130Dynamic Convolutional Filters
0.45
oracle success
No paper
#131rcm_test
0.44
oracle success
No paper
#132ai like samurai with PNasNet5Large
0.43
oracle success
No paper
#133base_1
0.43
oracle success
No paper
#134AnonymousTeam
0.41
oracle success
No paper
#135fuse_1
0.4
oracle success
No paper
#136base_0
0.4
oracle success
No paper
#137test_an
0.34
oracle success
No paper
#138Look Before You Leap
0.32
oracle success
No paper
#139X-Modal
0.32
oracle success
No paper
#140Seq2Seq Baseline
0.27
oracle success
No paper
#141zhangyong
0.26
oracle success
No paper
#142zy123
0.24
oracle success
No paper
#143zzzzzzzz55768
0.19
oracle success
No paper
#144Random Agent
0.18
oracle success
No paper
#14515458
0
oracle success
No paper