Cross-Modal Retrieval on Flickr30k

Metric: Text-to-image R@1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	Text-to-image R@1▼	Extra Data	Paper	Date↕	Code
1	ERNIE-ViL 2.0	93.3	Yes	ERNIE-ViL 2.0: Multi-view Contrastive Learning f...	2022-09-30	Code
2	X2-VLM (large)	91.8	Yes	X$^2$-VLM: All-In-One Pre-trained Model For Visi...	2022-11-22	Code
3	VAST	91	Yes	VAST: A Vision-Audio-Subtitle-Text Omni-Modality...	2023-05-29	Code
4	X2-VLM (base)	90.4	Yes	X$^2$-VLM: All-In-One Pre-trained Model For Visi...	2022-11-22	Code
5	BEiT-3	90.3	Yes	Image as a Foreign Language: BEiT Pretraining fo...	2022-08-22	Code
6	OmniVL (14M)	87.9	Yes	OmniVL:One Foundation Model for Image-Language a...	2022-09-15	-
7	X-VLM (base)	86.9	Yes	Multi-Grained Vision Language Pre-Training: Alig...	2021-11-16	Code
8	Aurora (ours, r=128)	86.8	Yes	-	-	-
9	VSE-Gradient	86.3	Yes	Dissecting Deep Metric Learning Losses for Image...	2022-10-21	Code
10	ALIGN	84.9	Yes	Scaling Up Visual and Vision-Language Representa...	2021-02-11	Code
11	IAIS	76.86	Yes	Learning Relation Alignment for Calibrated Cross...	2021-05-28	Code
12	ViSTA	75.8	Yes	ViSTA: Vision and Scene Text Aggregation for Cro...	2022-03-31	-
13	3SHNet	69.5	No	3SHNet: Boosting Image-Sentence Retrieval via Vi...	2024-04-26	Code
14	DSMD	68.4	No	Dynamic Self-adaptive Multiscale Distillation fr...	2024-04-16	Code
15	ViLT-B/32	64.4	Yes	ViLT: Vision-and-Language Transformer Without Co...	2021-02-05	Code
16	RCAR	62.6	No	Plug-and-Play Regulators for Image-Text Matching	2023-03-23	Code
17	NAPReg	60	No	-	-	Code
18	SGRAF	58.5	No	Similarity Reasoning and Filtration for Image-Te...	2021-01-05	Code
19	GSMN	57.4	No	Graph Structured Network for Image-Text Matching	2020-04-01	Code
20	Pearl	54.98	No	-	-	-
21	IMRAM	53.9	No	IMRAM: Iterative Matching with Recurrent Attenti...	2020-03-08	Code
22	SCAN	48.6	No	Stacked Cross Attention for Image-Text Matching	2018-03-21	Code
23	SCO (ResNet)	41.1	No	Learning Semantic Concepts and Order for Image a...	2017-12-06	-
24	VSE++ (ResNet)	39.6	No	VSE++: Improving Visual-Semantic Embeddings with...	2017-07-18	Code
25	Dual-Path (ResNet)	39.1	No	Dual-Path Convolutional Image-Text Embeddings wi...	2017-11-15	Code
26	CMPL (ResNet)	37.3	No	-	-	Code

#1ERNIE-ViL 2.0SOTA
93.3
Text-to-image R@1· Extra Data· 2022-09-30
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training Code
#2X2-VLM (large)
91.8
Text-to-image R@1· Extra Data· 2022-11-22
X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks Code
#3VAST
91
Text-to-image R@1· Extra Data· 2023-05-29
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Code
#4X2-VLM (base)
90.4
Text-to-image R@1· Extra Data· 2022-11-22
X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks Code
#5BEiT-3SOTA
90.3
Text-to-image R@1· Extra Data· 2022-08-22
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Code
#6OmniVL (14M)
87.9
Text-to-image R@1· Extra Data· 2022-09-15
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks
#7X-VLM (base)SOTA
86.9
Text-to-image R@1· Extra Data· 2021-11-16
Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts Code
#8Aurora (ours, r=128)
86.8
Text-to-image R@1· Extra Data
No paper
#9VSE-Gradient
86.3
Text-to-image R@1· Extra Data· 2022-10-21
Dissecting Deep Metric Learning Losses for Image-Text Retrieval Code
#10ALIGNSOTA
84.9
Text-to-image R@1· Extra Data· 2021-02-11
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Code
#11IAIS
76.86
Text-to-image R@1· Extra Data· 2021-05-28
Learning Relation Alignment for Calibrated Cross-modal Retrieval Code
#12ViSTA
75.8
Text-to-image R@1· Extra Data· 2022-03-31
ViSTA: Vision and Scene Text Aggregation for Cross-Modal Retrieval
#133SHNet
69.5
Text-to-image R@1· 2024-04-26
3SHNet: Boosting Image-Sentence Retrieval via Visual Semantic-Spatial Self-Highlighting Code
#14DSMD
68.4
Text-to-image R@1· 2024-04-16
Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Representation Learning Code
#15ViLT-B/32SOTA
64.4
Text-to-image R@1· Extra Data· 2021-02-05
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision Code
#16RCAR
62.6
Text-to-image R@1· 2023-03-23
Plug-and-Play Regulators for Image-Text Matching Code
#17NAPReg
60
Text-to-image R@1
No paperCode
#18SGRAFSOTA
58.5
Text-to-image R@1· 2021-01-05
Similarity Reasoning and Filtration for Image-Text Matching Code
#19GSMNSOTA
57.4
Text-to-image R@1· 2020-04-01
Graph Structured Network for Image-Text Matching Code
#20Pearl
54.98
Text-to-image R@1
No paper
#21IMRAMSOTA
53.9
Text-to-image R@1· 2020-03-08
IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval Code
#22SCANSOTA
48.6
Text-to-image R@1· 2018-03-21
Stacked Cross Attention for Image-Text Matching Code
#23SCO (ResNet)SOTA
41.1
Text-to-image R@1· 2017-12-06
Learning Semantic Concepts and Order for Image and Sentence Matching
#24VSE++ (ResNet)SOTA
39.6
Text-to-image R@1· 2017-07-18
VSE++: Improving Visual-Semantic Embeddings with Hard Negatives Code
#25Dual-Path (ResNet)
39.1
Text-to-image R@1· 2017-11-15
Dual-Path Convolutional Image-Text Embeddings with Instance Loss Code
#26CMPL (ResNet)
37.3
Text-to-image R@1
No paperCode