Video on MSVD

Metric: text-to-video R@1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	text-to-video R@1▼	Extra Data	Paper	Date↕	Code
1	InternVideo2-6B	61.4	Yes	InternVideo2: Scaling Foundation Models for Mult...	2024-03-22	Code
2	HunYuan_tvr (huge)	59	Yes	Tencent Text-Video Retrieval: Hierarchical Cross...	2022-04-07	-
3	InternVideo	58.4	Yes	InternVideo: General Video Foundation Models via...	2022-12-06	Code
4	HunYuan_tvr	58.2	Yes	Tencent Text-Video Retrieval: Hierarchical Cross...	2022-04-07	-
5	vid-TLDR (UMT-L)	57.9	Yes	vid-TLDR: Training Free Token merging for Light-...	2024-03-20	Code
6	VLAB	57.5	Yes	VLAB: Enhancing Video Language Pre-training by F...	2023-05-22	-
7	MDMMT-2	56.8	Yes	MDMMT-2: Multidomain Multimodal Transformer for ...	2022-03-14	-
8	Side4Video	56.1	No	Side4Video: Spatial-Temporal Side Network for Me...	2023-11-27	Code
9	CAMoE	51.8	Yes	Improving Video-Text Retrieval by Multi-Stream C...	2021-09-09	Code
10	Cap4Video	51.8	No	Cap4Video: What Can Auxiliary Captions Do for Te...	2022-12-31	Code
11	CenterCLIP (ViT-B/16)	50.6	Yes	CenterCLIP: Token Clustering for Efficient Text-...	2022-05-02	Code
12	X-CLIP	50.4	No	X-CLIP: End-to-End Multi-grained Contrastive Lea...	2022-07-15	Code
13	DMAE (ViT-B/32)	48.7	No	Dual-Modal Attention-Enhanced Text-Video Retriev...	2023-09-20	Code
14	QB-Norm+CLIP2Video	48	Yes	Cross Modal Retrieval with Querybank Normalisation	2021-12-23	Code
15	DiffusionRet+QB-Norm	47.9	No	DiffusionRet: Generative Text-Video Retrieval wi...	2023-03-17	Code
16	PAU	47.3	No	Prototype-based Aleatoric Uncertainty Quantifica...	2023-09-29	Code
17	X-Pool	47.2	Yes	X-Pool: Cross-Modal Language-Video Attention for...	2022-03-28	Code
18	DiffusionRet	46.6	No	DiffusionRet: Generative Text-Video Retrieval wi...	2023-03-17	Code
19	CLIP4Clip	46.2	Yes	CLIP4Clip: An Empirical Study of CLIP for End to...	2021-04-18	Code
20	LAFF	45.4	No	Lightweight Attentional Feature Fusion: A New Ba...	2021-12-03	Code
21	CLIP	37	No	A Straightforward Framework For Video Retrieval ...	2021-02-24	Code
22	FROZEN	33.7	Yes	Frozen in Time: A Joint Video and Image Encoder ...	2021-04-01	Code
23	SSML	20.3	No	Noise Estimation Using Density Estimation for Se...	2020-03-06	Code
24	Collaborative Experts	19.8	No	Use What You Have: Video Retrieval Using Represe...	2019-07-31	Code

#1InternVideo2-6BSOTA
61.4
text-to-video R@1· Extra Data· 2024-03-22
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding Code
#2HunYuan_tvr (huge)SOTA
59
text-to-video R@1· Extra Data· 2022-04-07
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations
#3InternVideo
58.4
text-to-video R@1· Extra Data· 2022-12-06
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Code
#4HunYuan_tvr
58.2
text-to-video R@1· Extra Data· 2022-04-07
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations
#5vid-TLDR (UMT-L)
57.9
text-to-video R@1· Extra Data· 2024-03-20
vid-TLDR: Training Free Token merging for Light-weight Video Transformer Code
#6VLAB
57.5
text-to-video R@1· Extra Data· 2023-05-22
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending
#7MDMMT-2SOTA
56.8
text-to-video R@1· Extra Data· 2022-03-14
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization
#8Side4Video
56.1
text-to-video R@1· 2023-11-27
Side4Video: Spatial-Temporal Side Network for Memory-Efficient Image-to-Video Transfer Learning Code
#9CAMoESOTA
51.8
text-to-video R@1· Extra Data· 2021-09-09
Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Loss Code
#10Cap4Video
51.8
text-to-video R@1· 2022-12-31
Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?Code
#11CenterCLIP (ViT-B/16)
50.6
text-to-video R@1· Extra Data· 2022-05-02
CenterCLIP: Token Clustering for Efficient Text-Video Retrieval Code
#12X-CLIP
50.4
text-to-video R@1· 2022-07-15
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Code
#13DMAE (ViT-B/32)
48.7
text-to-video R@1· 2023-09-20
Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning Code
#14QB-Norm+CLIP2Video
48
text-to-video R@1· Extra Data· 2021-12-23
Cross Modal Retrieval with Querybank Normalisation Code
#15DiffusionRet+QB-Norm
47.9
text-to-video R@1· 2023-03-17
DiffusionRet: Generative Text-Video Retrieval with Diffusion Model Code
#16PAU
47.3
text-to-video R@1· 2023-09-29
Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval Code
#17X-Pool
47.2
text-to-video R@1· Extra Data· 2022-03-28
X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval Code
#18DiffusionRet
46.6
text-to-video R@1· 2023-03-17
DiffusionRet: Generative Text-Video Retrieval with Diffusion Model Code
#19CLIP4ClipSOTA
46.2
text-to-video R@1· Extra Data· 2021-04-18
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Code
#20LAFF
45.4
text-to-video R@1· 2021-12-03
Lightweight Attentional Feature Fusion: A New Baseline for Text-to-Video Retrieval Code
#21CLIPSOTA
37
text-to-video R@1· 2021-02-24
A Straightforward Framework For Video Retrieval Using CLIP Code
#22FROZEN
33.7
text-to-video R@1· Extra Data· 2021-04-01
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Code
#23SSMLSOTA
20.3
text-to-video R@1· 2020-03-06
Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learning Code
#24Collaborative ExpertsSOTA
19.8
text-to-video R@1· 2019-07-31
Use What You Have: Video Retrieval Using Representations From Collaborative Experts Code