Video on ActivityNet

Metric: text-to-video R@5 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	text-to-video R@5▼	Extra Data	Paper	Date↕	Code
1	VAST	90.9	Yes	VAST: A Vision-Audio-Subtitle-Text Omni-Modality...	2023-05-29	Code
2	VALOR	90.8	Yes	VALOR: Vision-Audio-Language Omni-Perception Pre...	2023-04-17	Code
3	UMT-L (ViT-L/16)	89.1	Yes	Unmasked Teacher: Towards Training-Efficient Vid...	2023-03-28	Code
4	vid-TLDR (UMT-L)	88.6	Yes	vid-TLDR: Training Free Token merging for Light-...	2024-03-20	Code
5	CLIP-ViP	85.7	Yes	CLIP-ViP: Adapting Pre-trained Image-Text Model ...	2022-09-14	Code
6	HunYuan_tvr	84.8	Yes	Tencent Text-Video Retrieval: Hierarchical Cross...	2022-04-07	-
7	VindLU	81.4	Yes	VindLU: A Recipe for Effective Video-and-Languag...	2022-12-09	Code
8	RTQ	81.4	No	RTQ: Rethinking Video-language Understanding Bas...	2023-12-01	Code
9	TESTA (ViT-B/16)	80.8	Yes	TESTA: Temporal-Spatial Token Aggregation for Lo...	2023-10-29	Code
10	DMAE (ViT-B/32)	80.7	No	Dual-Modal Attention-Enhanced Text-Video Retriev...	2023-09-20	Code
11	EMCL-Net++	78.7	No	Expectation-Maximization Contrastive Learning fo...	2022-11-21	Code
12	CAMoE	77.7	Yes	Improving Video-Text Retrieval by Multi-Stream C...	2021-09-09	Code
13	HiTeA	77.1	Yes	HiTeA: Hierarchical Temporal-Aware Video-Languag...	2022-12-30	-
14	CenterCLIP (ViT-B/16)	77	Yes	CenterCLIP: Token Clustering for Efficient Text-...	2022-05-02	Code
15	DiffusionRet	75.6	No	DiffusionRet: Generative Text-Video Retrieval wi...	2023-03-17	Code
16	Singularity	75.5	Yes	Revealing Single Frame Bias for Video-and-Langua...	2022-06-07	Code
17	X-CLIP	75.5	No	X-CLIP: End-to-End Multi-grained Contrastive Lea...	2022-07-15	Code
18	CLIP4Clip	73.4	No	CLIP4Clip: An Empirical Study of CLIP for End to...	2021-04-18	Code
19	HBI	73	No	Video-Text as Game Players: Hierarchical Banzhaf...	2023-03-25	Code
20	EMCL-Net	72.7	No	Expectation-Maximization Contrastive Learning fo...	2022-11-21	Code
21	MMT-Pretrained	61.4	Yes	Multi-modal Transformer for Video Retrieval	2020-07-21	Code
22	TACo	61.2	Yes	TACo: Token-aware Cascade Contrastive Learning f...	2021-08-23	-
23	Ours	59.1	No	Video and Text Matching with Conditioned Embeddi...	2021-10-21	Code
24	HD-VILA	57.4	No	Advancing High-Resolution Video-Language Represe...	2021-11-19	Code
25	MMT	54.2	No	Multi-modal Transformer for Video Retrieval	2020-07-21	Code
26	Collaborative Experts	47.7	No	Use What You Have: Video Retrieval Using Represe...	2019-07-31	Code

#1VASTSOTA
90.9
text-to-video R@5· Extra Data· 2023-05-29
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Code
#2VALORSOTA
90.8
text-to-video R@5· Extra Data· 2023-04-17
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Code
#3UMT-L (ViT-L/16)SOTA
89.1
text-to-video R@5· Extra Data· 2023-03-28
Unmasked Teacher: Towards Training-Efficient Video Foundation Models Code
#4vid-TLDR (UMT-L)
88.6
text-to-video R@5· Extra Data· 2024-03-20
vid-TLDR: Training Free Token merging for Light-weight Video Transformer Code
#5CLIP-ViPSOTA
85.7
text-to-video R@5· Extra Data· 2022-09-14
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Code
#6HunYuan_tvrSOTA
84.8
text-to-video R@5· Extra Data· 2022-04-07
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations
#7VindLU
81.4
text-to-video R@5· Extra Data· 2022-12-09
VindLU: A Recipe for Effective Video-and-Language Pretraining Code
#8RTQ
81.4
text-to-video R@5· 2023-12-01
RTQ: Rethinking Video-language Understanding Based on Image-text Model Code
#9TESTA (ViT-B/16)
80.8
text-to-video R@5· Extra Data· 2023-10-29
TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding Code
#10DMAE (ViT-B/32)
80.7
text-to-video R@5· 2023-09-20
Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning Code
#11EMCL-Net++
78.7
text-to-video R@5· 2022-11-21
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations Code
#12CAMoESOTA
77.7
text-to-video R@5· Extra Data· 2021-09-09
Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Loss Code
#13HiTeA
77.1
text-to-video R@5· Extra Data· 2022-12-30
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
#14CenterCLIP (ViT-B/16)
77
text-to-video R@5· Extra Data· 2022-05-02
CenterCLIP: Token Clustering for Efficient Text-Video Retrieval Code
#15DiffusionRet
75.6
text-to-video R@5· 2023-03-17
DiffusionRet: Generative Text-Video Retrieval with Diffusion Model Code
#16Singularity
75.5
text-to-video R@5· Extra Data· 2022-06-07
Revealing Single Frame Bias for Video-and-Language Learning Code
#17X-CLIP
75.5
text-to-video R@5· 2022-07-15
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Code
#18CLIP4ClipSOTA
73.4
text-to-video R@5· 2021-04-18
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Code
#19HBI
73
text-to-video R@5· 2023-03-25
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning Code
#20EMCL-Net
72.7
text-to-video R@5· 2022-11-21
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations Code
#21MMT-PretrainedSOTA
61.4
text-to-video R@5· Extra Data· 2020-07-21
Multi-modal Transformer for Video Retrieval Code
#22TACo
61.2
text-to-video R@5· Extra Data· 2021-08-23
TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment
#23Ours
59.1
text-to-video R@5· 2021-10-21
Video and Text Matching with Conditioned Embeddings Code
#24HD-VILA
57.4
text-to-video R@5· 2021-11-19
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Code
#25MMT
54.2
text-to-video R@5· 2020-07-21
Multi-modal Transformer for Video Retrieval Code
#26Collaborative ExpertsSOTA
47.7
text-to-video R@5· 2019-07-31
Use What You Have: Video Retrieval Using Representations From Collaborative Experts Code