Tasks
SotA
Datasets
Papers
Methods
Submit
About
SotA
/
Miscellaneous
/
Image Retrieval with Multi-Modal Query
/
Flickr30k
Image Retrieval with Multi-Modal Query on Flickr30k
Metric: Image-to-text R@5 (higher is better)
Leaderboard
Dataset
Loading chart...
Results
Submit a result
Hide extra data
Export CSV
Sort:
Image-to-text R@5 (best first)
Image-to-text R@5 (worst first)
Date (newest first)
Date (oldest first)
Model name (A→Z)
#
Model
↕
Image-to-text R@5
▼
Extra Data
Paper
Date
↕
Code
1
X2-VLM (large)
100
Yes
X$^2$-VLM: All-In-One Pre-trained Model For Visi...
2022-11-22
Code
2
X2-VLM (base)
100
Yes
X$^2$-VLM: All-In-One Pre-trained Model For Visi...
2022-11-22
Code
3
BEiT-3
100
Yes
Image as a Foreign Language: BEiT Pretraining fo...
2022-08-22
Code
4
ERNIE-ViL 2.0
100
Yes
ERNIE-ViL 2.0: Multi-view Contrastive Learning f...
2022-09-30
Code
5
Aurora (ours, r=128)
100
Yes
-
-
-
6
X-VLM (base)
100
Yes
Multi-Grained Vision Language Pre-Training: Alig...
2021-11-16
Code
7
OmniVL (14M)
99.9
Yes
OmniVL:One Foundation Model for Image-Language a...
2022-09-15
-
8
BEiT-3
99.9
No
Image as a Foreign Language: BEiT Pretraining fo...
2022-08-22
Code
9
ALIGN
99.8
Yes
Scaling Up Visual and Vision-Language Representa...
2021-02-11
Code
10
InternVL-G
99.7
No
InternVL: Scaling up Vision Foundation Models an...
2023-12-21
Code
11
VSE-Gradient
99.6
Yes
Dissecting Deep Metric Learning Losses for Image...
2022-10-21
Code
12
InternVL-C
99.6
No
InternVL: Scaling up Vision Foundation Models an...
2023-12-21
Code
13
CoCa
99.5
No
CoCa: Contrastive Captioners are Image-Text Foun...
2022-05-04
Code
14
COSMOS ViT-B/16
99.4
No
COSMOS: Cross-Modality Self-Distillation for Vis...
2024-12-02
Code
15
RO-ViT
99.4
No
Region-Aware Pretraining for Open-Vocabulary Obj...
2023-05-11
Code
16
OpenCLIP VIT-H/14
99.3
No
Reproducible scaling laws for contrastive langua...
2022-12-14
Code
17
M2-Encoder
99.2
Yes
M2-Encoder: Advancing Bilingual Image-Text Under...
2024-01-29
Code
18
VK-OOD
99.2
No
-
-
Code
19
ERNIE-ViL 2.0
99.1
No
ERNIE-ViL 2.0: Multi-view Contrastive Learning f...
2022-09-30
Code
20
Florence
99.1
No
Florence: A New Foundation Model for Computer Vi...
2021-11-22
Code
21
ALBEF
98.8
No
Align before Fuse: Vision and Language Represent...
2021-07-16
Code
22
COSMOS ViT-B/32
98.8
No
COSMOS: Cross-Modality Self-Distillation for Vis...
2024-12-02
Code
23
Flamingo
98.8
No
Flamingo: a Visual Language Model for Few-Shot L...
2022-04-29
Code
24
ALIGN
98.7
No
Scaling Up Visual and Vision-Language Representa...
2021-02-11
Code
25
CLIP
98.7
No
Learning Transferable Visual Models From Natural...
2021-02-26
Code
26
ViSTA
98.4
Yes
ViSTA: Vision and Scene Text Aggregation for Cro...
2022-03-31
-
27
IAIS
98.4
Yes
Learning Relation Alignment for Calibrated Cross...
2021-05-28
Code
28
PTP-BLIP (14M)
98.4
No
Position-guided Text Prompt for Vision-Language ...
2022-12-19
Code
29
3SHNet
98.2
No
3SHNet: Boosting Image-Sentence Retrieval via Vi...
2024-04-26
Code
30
AltCLIP
98
No
AltCLIP: Altering the Language Encoder in CLIP f...
2022-11-12
Code
31
ViLT-B/32
96.7
Yes
ViLT: Vision-and-Language Transformer Without Co...
2021-02-05
Code
32
RCAR
96
No
Plug-and-Play Regulators for Image-Text Matching
2023-03-23
Code
33
UNITER
95.7
No
UNITER: UNiversal Image-TExt Representation Lear...
2019-09-25
Code
34
DSMD
95.5
No
Dynamic Self-adaptive Multiscale Distillation fr...
2024-04-16
Code
35
GSMN
94.3
No
A Deep Local and Global Scene-Graph Matching for...
2021-06-04
Code
36
SGRAF
94.1
No
Similarity Reasoning and Filtration for Image-Te...
2021-01-05
Code
37
ViLT-B/32
93.6
No
ViLT: Vision-and-Language Transformer Without Co...
2021-02-05
Code
38
Pearl
93.4
No
-
-
-
39
IMRAM
93
No
IMRAM: Iterative Matching with Recurrent Attenti...
2020-03-08
Code
40
SCAN
90.3
No
Stacked Cross Attention for Image-Text Matching
2018-03-21
Code
41
ImageBERT
90.2
No
ImageBERT: Cross-modal Pre-training with Large-s...
2020-01-22
-
42
SCO (ResNet)
82
No
Learning Semantic Concepts and Order for Image a...
2017-12-06
-
43
Dual-Path (ResNet)
81.9
No
Dual-Path Convolutional Image-Text Embeddings wi...
2017-11-15
Code
44
VSE++ (ResNet)
80.5
No
VSE++: Improving Visual-Semantic Embeddings with...
2017-07-18
Code
45
CMPL (ResNet)
76.8
No
-
-
Code
#1
X2-VLM (large)
100
Image-to-text R@5
· Extra Data
· 2022-11-22
X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks
Code
#2
X2-VLM (base)
100
Image-to-text R@5
· Extra Data
· 2022-11-22
X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks
Code
#3
BEiT-3
100
Image-to-text R@5
· Extra Data
· 2022-08-22
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
Code
#4
ERNIE-ViL 2.0
100
Image-to-text R@5
· Extra Data
· 2022-09-30
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training
Code
#5
Aurora (ours, r=128)
100
Image-to-text R@5
· Extra Data
No paper
#6
X-VLM (base)
SOTA
100
Image-to-text R@5
· Extra Data
· 2021-11-16
Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts
Code
#7
OmniVL (14M)
99.9
Image-to-text R@5
· Extra Data
· 2022-09-15
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks
#8
BEiT-3
99.9
Image-to-text R@5
· 2022-08-22
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
Code
#9
ALIGN
SOTA
99.8
Image-to-text R@5
· Extra Data
· 2021-02-11
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Code
#10
InternVL-G
99.7
Image-to-text R@5
· 2023-12-21
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Code
#11
VSE-Gradient
99.6
Image-to-text R@5
· Extra Data
· 2022-10-21
Dissecting Deep Metric Learning Losses for Image-Text Retrieval
Code
#12
InternVL-C
99.6
Image-to-text R@5
· 2023-12-21
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Code
#13
CoCa
99.5
Image-to-text R@5
· 2022-05-04
CoCa: Contrastive Captioners are Image-Text Foundation Models
Code
#14
COSMOS ViT-B/16
99.4
Image-to-text R@5
· 2024-12-02
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training
Code
#15
RO-ViT
99.4
Image-to-text R@5
· 2023-05-11
Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformers
Code
#16
OpenCLIP VIT-H/14
99.3
Image-to-text R@5
· 2022-12-14
Reproducible scaling laws for contrastive language-image learning
Code
#17
M2-Encoder
99.2
Image-to-text R@5
· Extra Data
· 2024-01-29
M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining
Code
#18
VK-OOD
99.2
Image-to-text R@5
No paper
Code
#19
ERNIE-ViL 2.0
99.1
Image-to-text R@5
· 2022-09-30
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training
Code
#20
Florence
99.1
Image-to-text R@5
· 2021-11-22
Florence: A New Foundation Model for Computer Vision
Code
#21
ALBEF
98.8
Image-to-text R@5
· 2021-07-16
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Code
#22
COSMOS ViT-B/32
98.8
Image-to-text R@5
· 2024-12-02
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training
Code
#23
Flamingo
98.8
Image-to-text R@5
· 2022-04-29
Flamingo: a Visual Language Model for Few-Shot Learning
Code
#24
ALIGN
98.7
Image-to-text R@5
· 2021-02-11
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Code
#25
CLIP
98.7
Image-to-text R@5
· 2021-02-26
Learning Transferable Visual Models From Natural Language Supervision
Code
#26
ViSTA
98.4
Image-to-text R@5
· Extra Data
· 2022-03-31
ViSTA: Vision and Scene Text Aggregation for Cross-Modal Retrieval
#27
IAIS
98.4
Image-to-text R@5
· Extra Data
· 2021-05-28
Learning Relation Alignment for Calibrated Cross-modal Retrieval
Code
#28
PTP-BLIP (14M)
98.4
Image-to-text R@5
· 2022-12-19
Position-guided Text Prompt for Vision-Language Pre-training
Code
#29
3SHNet
98.2
Image-to-text R@5
· 2024-04-26
3SHNet: Boosting Image-Sentence Retrieval via Visual Semantic-Spatial Self-Highlighting
Code
#30
AltCLIP
98
Image-to-text R@5
· 2022-11-12
AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities
Code
#31
ViLT-B/32
SOTA
96.7
Image-to-text R@5
· Extra Data
· 2021-02-05
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
Code
#32
RCAR
96
Image-to-text R@5
· 2023-03-23
Plug-and-Play Regulators for Image-Text Matching
Code
#33
UNITER
SOTA
95.7
Image-to-text R@5
· 2019-09-25
UNITER: UNiversal Image-TExt Representation Learning
Code
#34
DSMD
95.5
Image-to-text R@5
· 2024-04-16
Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Representation Learning
Code
#35
GSMN
94.3
Image-to-text R@5
· 2021-06-04
A Deep Local and Global Scene-Graph Matching for Image-Text Retrieval
Code
#36
SGRAF
94.1
Image-to-text R@5
· 2021-01-05
Similarity Reasoning and Filtration for Image-Text Matching
Code
#37
ViLT-B/32
93.6
Image-to-text R@5
· 2021-02-05
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
Code
#38
Pearl
93.4
Image-to-text R@5
No paper
#39
IMRAM
93
Image-to-text R@5
· 2020-03-08
IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval
Code
#40
SCAN
SOTA
90.3
Image-to-text R@5
· 2018-03-21
Stacked Cross Attention for Image-Text Matching
Code
#41
ImageBERT
90.2
Image-to-text R@5
· 2020-01-22
ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data
#42
SCO (ResNet)
SOTA
82
Image-to-text R@5
· 2017-12-06
Learning Semantic Concepts and Order for Image and Sentence Matching
#43
Dual-Path (ResNet)
SOTA
81.9
Image-to-text R@5
· 2017-11-15
Dual-Path Convolutional Image-Text Embeddings with Instance Loss
Code
#44
VSE++ (ResNet)
SOTA
80.5
Image-to-text R@5
· 2017-07-18
VSE++: Improving Visual-Semantic Embeddings with Hard Negatives
Code
#45
CMPL (ResNet)
76.8
Image-to-text R@5
No paper
Code