Image Retrieval with Multi-Modal Query on Flickr30k

Metric: Text-to-image R@1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

#	Model↕	Text-to-image R@1▼	Extra Data	Paper	Date↕	Code
1	ERNIE-ViL 2.0	93.3	Yes	ERNIE-ViL 2.0: Multi-view Contrastive Learning f...	2022-09-30	Code
2	M2-Encoder	92.2	Yes	M2-Encoder: Advancing Bilingual Image-Text Under...	2024-01-29	Code
3	X2-VLM (large)	91.8	Yes	X$^2$-VLM: All-In-One Pre-trained Model For Visi...	2022-11-22	Code
4	VAST	91	Yes	VAST: A Vision-Audio-Subtitle-Text Omni-Modality...	2023-05-29	Code
5	X2-VLM (base)	90.4	Yes	X$^2$-VLM: All-In-One Pre-trained Model For Visi...	2022-11-22	Code
6	VAST	90.4	Yes	VAST: A Vision-Audio-Subtitle-Text Omni-Modality...	2023-05-29	Code
7	BEiT-3	90.3	Yes	Image as a Foreign Language: BEiT Pretraining fo...	2022-08-22	Code
8	OmniVL (14M)	87.9	Yes	OmniVL:One Foundation Model for Image-Language a...	2022-09-15	-
9	X-VLM (base)	86.9	Yes	Multi-Grained Vision Language Pre-Training: Alig...	2021-11-16	Code
10	Aurora (ours, r=128)	86.8	Yes	-	-	-
11	VSE-Gradient	86.3	Yes	Dissecting Deep Metric Learning Losses for Image...	2022-10-21	Code
12	InternVL-G	85	No	InternVL: Scaling up Vision Foundation Models an...	2023-12-21	Code
13	ALIGN	84.9	Yes	Scaling Up Visual and Vision-Language Representa...	2021-02-11	Code
14	InternVL-C	81.7	No	InternVL: Scaling up Vision Foundation Models an...	2023-12-21	Code
15	BEiT-3	81.5	No	Image as a Foreign Language: BEiT Pretraining fo...	2022-08-22	Code
16	RO-ViT	80.7	No	Region-Aware Pretraining for Open-Vocabulary Obj...	2023-05-11	Code
17	CoCa	80.4	No	CoCa: Contrastive Captioners are Image-Text Foun...	2022-05-04	Code
18	COSMOS ViT-B/16	80.3	No	COSMOS: Cross-Modality Self-Distillation for Vis...	2024-12-02	Code
19	Flamingo	79.5	No	Flamingo: a Visual Language Model for Few-Shot L...	2022-04-29	Code
20	ERNIE-ViL 2.0	77.4	No	ERNIE-ViL 2.0: Multi-view Contrastive Learning f...	2022-09-30	Code
21	VK-OOD	77.2	No	-	-	Code
22	IAIS	76.86	Yes	Learning Relation Alignment for Calibrated Cross...	2021-05-28	Code
23	ALBEF	76.8	No	Align before Fuse: Vision and Language Represent...	2021-07-16	Code
24	Florence	76.7	No	Florence: A New Foundation Model for Computer Vi...	2021-11-22	Code
25	COSMOS ViT-B/32	76.1	No	COSMOS: Cross-Modality Self-Distillation for Vis...	2024-12-02	Code
26	ViSTA	75.8	Yes	ViSTA: Vision and Scene Text Aggregation for Cro...	2022-03-31	-
27	ALIGN	75.7	No	Scaling Up Visual and Vision-Language Representa...	2021-02-11	Code
28	PTP-BLIP (14M)	73.1	No	Position-guided Text Prompt for Vision-Language ...	2022-12-19	Code
29	AltCLIP	72.5	No	AltCLIP: Altering the Language Encoder in CLIP f...	2022-11-12	Code
30	3SHNet	69.5	No	3SHNet: Boosting Image-Sentence Retrieval via Vi...	2024-04-26	Code
31	CLIP	68.7	No	Learning Transferable Visual Models From Natural...	2021-02-26	Code
32	DSMD	68.4	No	Dynamic Self-adaptive Multiscale Distillation fr...	2024-04-16	Code
33	UNITER	66.2	No	UNITER: UNiversal Image-TExt Representation Lear...	2019-09-25	Code
34	ViLT-B/32	64.4	Yes	ViLT: Vision-and-Language Transformer Without Co...	2021-02-05	Code
35	RCAR	62.6	No	Plug-and-Play Regulators for Image-Text Matching	2023-03-23	Code
36	NAPReg	60	No	-	-	Code
37	SGRAF	58.5	No	Similarity Reasoning and Filtration for Image-Te...	2021-01-05	Code
38	GSMN	57.4	No	Graph Structured Network for Image-Text Matching	2020-04-01	Code
39	ViLT-B/32	55	No	ViLT: Vision-and-Language Transformer Without Co...	2021-02-05	Code
40	Pearl	54.98	No	-	-	-
41	ImageBERT	54.3	No	ImageBERT: Cross-modal Pre-training with Large-s...	2020-01-22	-
42	IMRAM	53.9	No	IMRAM: Iterative Matching with Recurrent Attenti...	2020-03-08	Code
43	SCAN	48.6	No	Stacked Cross Attention for Image-Text Matching	2018-03-21	Code
44	SCO (ResNet)	41.1	No	Learning Semantic Concepts and Order for Image a...	2017-12-06	-
45	VSE++ (ResNet)	39.6	No	VSE++: Improving Visual-Semantic Embeddings with...	2017-07-18	Code
46	Dual-Path (ResNet)	39.1	No	Dual-Path Convolutional Image-Text Embeddings wi...	2017-11-15	Code
47	CMPL (ResNet)	37.3	No	-	-	Code