Video Retrieval on LSMDC

Metric: text-to-video R@1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	text-to-video R@1▼	Extra Data	Paper	Date↕	Code
1	InternVideo2-6B	46.4	Yes	InternVideo2: Scaling Foundation Models for Mult...	2024-03-22	Code
2	vid-TLDR (UMT-L)	43.1	Yes	vid-TLDR: Training Free Token merging for Light-...	2024-03-20	Code
3	UMT-L (ViT-L/16)	43	Yes	Unmasked Teacher: Towards Training-Efficient Vid...	2023-03-28	Code
4	HunYuan_tvr (huge)	40.4	Yes	Tencent Text-Video Retrieval: Hierarchical Cross...	2022-04-07	-
5	COSA	39.4	Yes	COSA: Concatenated Sample Pretrained Vision-Lang...	2023-06-15	Code
6	mPLUG-2	34.4	Yes	mPLUG-2: A Modularized Multi-modal Foundation Mo...	2023-02-01	Code
7	VALOR	34.2	Yes	VALOR: Vision-Audio-Language Omni-Perception Pre...	2023-04-17	Code
8	InternVideo	34	Yes	InternVideo: General Video Foundation Models via...	2022-12-06	Code
9	CLIP-ViP	30.7	Yes	CLIP-ViP: Adapting Pre-trained Image-Text Model ...	2022-09-14	Code
10	HunYuan_tvr	29.7	Yes	Tencent Text-Video Retrieval: Hierarchical Cross...	2022-04-07	-
11	STAN	29.2	Yes	Revisiting Temporal Modeling for CLIP-based Imag...	2023-01-26	Code
12	HiTeA	28.7	Yes	HiTeA: Hierarchical Temporal-Aware Video-Languag...	2022-12-30	-
13	MDMMT-2	26.9	Yes	MDMMT-2: Multidomain Multimodal Transformer for ...	2022-03-14	-
14	X-CLIP	26.1	No	X-CLIP: End-to-End Multi-grained Contrastive Lea...	2022-07-15	Code
15	EMCL-Net++	25.9	No	Expectation-Maximization Contrastive Learning fo...	2022-11-21	Code
16	CAMoE	25.9	Yes	Improving Video-Text Retrieval by Multi-Stream C...	2021-09-09	Code
17	X-Pool	25.2	Yes	X-Pool: Cross-Modal Language-Video Attention for...	2022-03-28	Code
18	Clover	24.8	No	Clover: Towards A Unified Video-Language Alignme...	2022-07-16	Code
19	DiffusionRet	24.4	No	DiffusionRet: Generative Text-Video Retrieval wi...	2023-03-17	Code
20	CenterCLIP (ViT-B/16)	24.2	Yes	CenterCLIP: Token Clustering for Efficient Text-...	2022-05-02	Code
21	VIOLETv2	24	No	An Empirical Study of End-to-End Video-Language ...	2022-09-04	Code
22	EMCL-Net	23.9	No	Expectation-Maximization Contrastive Learning fo...	2022-11-21	Code
23	QB-Norm+CLIP4Clip	22.4	Yes	Cross Modal Retrieval with Querybank Normalisation	2021-12-23	Code
24	CLIP4Clip	21.6	Yes	CLIP4Clip: An Empirical Study of CLIP for End to...	2021-04-18	Code
25	MDMMT	18.8	Yes	MDMMT: Multidomain Multimodal Transformer for Vi...	2021-03-19	Code
26	HD-VILA	17.4	No	Advancing High-Resolution Video-Language Represe...	2021-11-19	Code
27	FROZEN	15	Yes	Frozen in Time: A Joint Video and Image Encoder ...	2021-04-01	Code
28	Ours	14.9	No	Video and Text Matching with Conditioned Embeddi...	2021-10-21	Code
29	MMT-Pretrained	13.5	Yes	Multi-modal Transformer for Video Retrieval	2020-07-21	Code
30	MMT	13.2	No	Multi-modal Transformer for Video Retrieval	2020-07-21	Code
31	CLIP	11.3	No	A Straightforward Framework For Video Retrieval ...	2021-02-24	Code
32	Collaborative Experts	11.2	No	Use What You Have: Video Retrieval Using Represe...	2019-07-31	Code
33	MoEE	10.1	Yes	Learning a Text-Video Embedding from Incomplete ...	2018-04-07	Code
34	JSFusion	9.1	No	A Joint Sequence Fusion Model for Video Question...	2018-08-07	Code
35	Large-Scale Discriminative Clustering	7.3	No	Learning from Video and Text via Large-Scale Dis...	2017-07-27	Code
36	Text-Video Embedding	7.2	No	HowTo100M: Learning a Text-Video Embedding by Wa...	2019-06-07	Code
37	CT-SAN	5.1	No	End-to-end Concept Word Detection for Video Capt...	2016-10-10	-

#1InternVideo2-6BSOTA
46.4
text-to-video R@1· Extra Data· 2024-03-22
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding Code
#2vid-TLDR (UMT-L)SOTA
43.1
text-to-video R@1· Extra Data· 2024-03-20
vid-TLDR: Training Free Token merging for Light-weight Video Transformer Code
#3UMT-L (ViT-L/16)SOTA
43
text-to-video R@1· Extra Data· 2023-03-28
Unmasked Teacher: Towards Training-Efficient Video Foundation Models Code
#4HunYuan_tvr (huge)SOTA
40.4
text-to-video R@1· Extra Data· 2022-04-07
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations
#5COSA
39.4
text-to-video R@1· Extra Data· 2023-06-15
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model Code
#6mPLUG-2
34.4
text-to-video R@1· Extra Data· 2023-02-01
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video Code
#7VALOR
34.2
text-to-video R@1· Extra Data· 2023-04-17
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Code
#8InternVideo
34
text-to-video R@1· Extra Data· 2022-12-06
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Code
#9CLIP-ViP
30.7
text-to-video R@1· Extra Data· 2022-09-14
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Code
#10HunYuan_tvr
29.7
text-to-video R@1· Extra Data· 2022-04-07
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations
#11STAN
29.2
text-to-video R@1· Extra Data· 2023-01-26
Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferring Code
#12HiTeA
28.7
text-to-video R@1· Extra Data· 2022-12-30
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
#13MDMMT-2SOTA
26.9
text-to-video R@1· Extra Data· 2022-03-14
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization
#14X-CLIP
26.1
text-to-video R@1· 2022-07-15
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Code
#15EMCL-Net++
25.9
text-to-video R@1· 2022-11-21
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations Code
#16CAMoESOTA
25.9
text-to-video R@1· Extra Data· 2021-09-09
Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Loss Code
#17X-Pool
25.2
text-to-video R@1· Extra Data· 2022-03-28
X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval Code
#18Clover
24.8
text-to-video R@1· 2022-07-16
Clover: Towards A Unified Video-Language Alignment and Fusion Model Code
#19DiffusionRet
24.4
text-to-video R@1· 2023-03-17
DiffusionRet: Generative Text-Video Retrieval with Diffusion Model Code
#20CenterCLIP (ViT-B/16)
24.2
text-to-video R@1· Extra Data· 2022-05-02
CenterCLIP: Token Clustering for Efficient Text-Video Retrieval Code
#21VIOLETv2
24
text-to-video R@1· 2022-09-04
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling Code
#22EMCL-Net
23.9
text-to-video R@1· 2022-11-21
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations Code
#23QB-Norm+CLIP4Clip
22.4
text-to-video R@1· Extra Data· 2021-12-23
Cross Modal Retrieval with Querybank Normalisation Code
#24CLIP4ClipSOTA
21.6
text-to-video R@1· Extra Data· 2021-04-18
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Code
#25MDMMTSOTA
18.8
text-to-video R@1· Extra Data· 2021-03-19
MDMMT: Multidomain Multimodal Transformer for Video Retrieval Code
#26HD-VILA
17.4
text-to-video R@1· 2021-11-19
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Code
#27FROZEN
15
text-to-video R@1· Extra Data· 2021-04-01
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Code
#28Ours
14.9
text-to-video R@1· 2021-10-21
Video and Text Matching with Conditioned Embeddings Code
#29MMT-PretrainedSOTA
13.5
text-to-video R@1· Extra Data· 2020-07-21
Multi-modal Transformer for Video Retrieval Code
#30MMT
13.2
text-to-video R@1· 2020-07-21
Multi-modal Transformer for Video Retrieval Code
#31CLIP
11.3
text-to-video R@1· 2021-02-24
A Straightforward Framework For Video Retrieval Using CLIP Code
#32Collaborative ExpertsSOTA
11.2
text-to-video R@1· 2019-07-31
Use What You Have: Video Retrieval Using Representations From Collaborative Experts Code
#33MoEESOTA
10.1
text-to-video R@1· Extra Data· 2018-04-07
Learning a Text-Video Embedding from Incomplete and Heterogeneous Data Code
#34JSFusion
9.1
text-to-video R@1· 2018-08-07
A Joint Sequence Fusion Model for Video Question Answering and Retrieval Code
#35Large-Scale Discriminative ClusteringSOTA
7.3
text-to-video R@1· 2017-07-27
Learning from Video and Text via Large-Scale Discriminative Clustering Code
#36Text-Video Embedding
7.2
text-to-video R@1· 2019-06-07
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Code
#37CT-SANSOTA
5.1
text-to-video R@1· 2016-10-10
End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answering