Image Retrieval with Multi-Modal Query on COCO 2014

Metric: Image-to-text R@1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

#	Model↕	Image-to-text R@1▼	Extra Data	Paper	Date↕	Code
1	BEiT-3	84.8	Yes	Image as a Foreign Language: BEiT Pretraining fo...	2022-08-22	Code
2	X2-VLM (large)	84.4	Yes	X$^2$-VLM: All-In-One Pre-trained Model For Visi...	2022-11-22	Code
3	XFM (base)	84.2	Yes	Toward Building General Foundation Models for La...	2023-01-12	Code
4	X2-VLM (base)	83.5	Yes	X$^2$-VLM: All-In-One Pre-trained Model For Visi...	2022-11-22	Code
5	OmniVL (14M)	82.1	Yes	OmniVL:One Foundation Model for Image-Language a...	2022-09-15	-
6	Florence	81.8	Yes	Florence: A New Foundation Model for Computer Vi...	2021-11-22	Code
7	PTP-BLIP (14M)	81.5	Yes	Position-guided Text Prompt for Vision-Language ...	2022-12-19	Code
8	VSE-Gradient	81.4	Yes	Dissecting Deep Metric Learning Losses for Image...	2022-10-21	Code
9	X-VLM (base)	81.2	Yes	Multi-Grained Vision Language Pre-Training: Alig...	2021-11-16	Code
10	VK-OOD	80.7	Yes	-	-	Code
11	Aurora (ours, r=128)	80.7	Yes	-	-	-
12	ALBEF	77.6	Yes	Align before Fuse: Vision and Language Represent...	2021-07-16	Code
13	ERNIE-ViL 2.0	77.4	Yes	ERNIE-ViL 2.0: Multi-view Contrastive Learning f...	2022-09-30	Code
14	ALIGN	77	Yes	Scaling Up Visual and Vision-Language Representa...	2021-02-11	Code
15	METER	76.16	Yes	An Empirical Study of Training End-to-End Vision...	2021-11-03	Code
16	TCL	75.6	Yes	Vision-Language Pre-Training with Triple Contras...	2022-02-21	Code
17	InternVL-G	74.9	No	InternVL: Scaling up Vision Foundation Models an...	2023-12-21	Code
18	Oscar	73.5	Yes	Oscar: Object-Semantics Aligned Pre-training for...	2020-04-13	Code
19	M2-Encoder	72.8	No	M2-Encoder: Advancing Bilingual Image-Text Under...	2024-01-29	Code
20	TCL	71.4	No	Vision-Language Pre-Training with Triple Contras...	2022-02-21	Code
21	MaMMUT (ours)	70.7	No	MaMMUT: A Simple Architecture for Joint Learning...	2023-03-29	Code
22	InternVL-C	70.6	No	InternVL: Scaling up Vision Foundation Models an...	2023-12-21	Code
23	PTP-BLIP	69.7	No	Position-guided Text Prompt for Vision-Language ...	2022-12-19	Code
24	ViSTA	68.9	Yes	ViSTA: Vision and Scene Text Aggregation for Cro...	2022-03-31	-
25	RO-ViT	68.9	No	Region-Aware Pretraining for Open-Vocabulary Obj...	2023-05-11	Code
26	ALBEF	68.7	No	Align before Fuse: Vision and Language Represent...	2021-07-16	Code
27	COSMOS ViT-B/16	68	No	COSMOS: Cross-Modality Self-Distillation for Vis...	2024-12-02	Code
28	3SHNet	67.9	No	3SHNet: Boosting Image-Sentence Retrieval via Vi...	2024-04-26	Code
29	CoCa	66.3	No	CoCa: Contrastive Captioners are Image-Text Foun...	2022-05-04	Code
30	Flamingo	65.9	No	Flamingo: a Visual Language Model for Few-Shot L...	2022-04-29	Code
31	ALADIN	64.9	Yes	ALADIN: Distilling Fine-grained Alignment Scores...	2022-07-29	Code
32	Florence	64.7	No	Florence: A New Foundation Model for Computer Vi...	2021-11-22	Code
33	COSMOS ViT-B/32	64.3	No	COSMOS: Cross-Modality Self-Distillation for Vis...	2024-12-02	Code
34	ERNIE-ViL 2.0	63.1	No	ERNIE-ViL 2.0: Multi-view Contrastive Learning f...	2022-09-30	Code
35	ViLT-B/32	61.5	Yes	ViLT: Vision-and-Language Transformer Without Co...	2021-02-05	Code
36	RCAR	61.3	No	Plug-and-Play Regulators for Image-Text Matching	2023-03-23	Code
37	NAPReg	59.8	No	-	-	Code
38	ALIGN	58.6	No	Scaling Up Visual and Vision-Language Representa...	2021-02-11	Code
39	CLIP	58.4	No	Learning Transferable Visual Models From Natural...	2021-02-26	Code
40	SGRAF	57.8	No	Similarity Reasoning and Filtration for Image-Te...	2021-01-05	Code
41	ViLT-B/32	56.5	No	ViLT: Vision-and-Language Transformer Without Co...	2021-02-05	Code
42	LILE	55.6	No	LILE: Look In-Depth before Looking Elsewhere -- ...	2022-03-02	-
43	IMRAM	53.7	No	IMRAM: Iterative Matching with Recurrent Attenti...	2020-03-08	Code
44	VSRN	53	No	Visual Semantic Reasoning for Image-Text Matching	2019-09-06	Code
45	SCAN	50.4	No	Stacked Cross Attention for Image-Text Matching	2018-03-21	Code
46	DSMD	48	No	Dynamic Self-adaptive Multiscale Distillation fr...	2024-04-16	Code
47	PVSE	45.2	No	Polysemous Visual-Semantic Embedding for Cross-M...	2019-06-11	Code
48	ImageBERT	44	No	ImageBERT: Cross-modal Pre-training with Large-s...	2020-01-22	-
49	SCO (ResNet)	42.8	No	Learning Semantic Concepts and Order for Image a...	2017-12-06	-
50	Dual-Path (ResNet)	41.2	No	Deep Visual-Semantic Alignments for Generating I...	2014-12-07	Code
51	dfdf	0	No	-	-	-