Video Retrieval on ActivityNet

Metric: text-to-video R@1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	text-to-video R@1▼	Extra Data	Paper	Date↕	Code
1	InternVideo2-6B	74.1	Yes	InternVideo2: Scaling Foundation Models for Mult...	2024-03-22	Code
2	VAST	70.5	Yes	VAST: A Vision-Audio-Subtitle-Text Omni-Modality...	2023-05-29	Code
3	VALOR	70.1	Yes	VALOR: Vision-Audio-Language Omni-Perception Pre...	2023-04-17	Code
4	GRAM	69.9	Yes	Gramian Multimodal Representation Learning and A...	2024-12-16	Code
5	COSA	67.3	Yes	COSA: Concatenated Sample Pretrained Vision-Lang...	2023-06-15	Code
6	UMT-L (ViT-L/16)	66.8	Yes	Unmasked Teacher: Towards Training-Efficient Vid...	2023-03-28	Code
7	vid-TLDR (UMT-L)	66.7	Yes	vid-TLDR: Training Free Token merging for Light-...	2024-03-20	Code
8	InternVideo	62.2	Yes	InternVideo: General Video Foundation Models via...	2022-12-06	Code
9	CLIP-ViP	61.4	Yes	CLIP-ViP: Adapting Pre-trained Image-Text Model ...	2022-09-14	Code
10	HunYuan_tvr	57.3	Yes	Tencent Text-Video Retrieval: Hierarchical Cross...	2022-04-07	-
11	VindLU	55	Yes	VindLU: A Recipe for Effective Video-and-Languag...	2022-12-09	Code
12	TESTA (ViT-B/16)	54.8	Yes	TESTA: Temporal-Spatial Token Aggregation for Lo...	2023-10-29	Code
13	RTQ	53.5	No	RTQ: Rethinking Video-language Understanding Bas...	2023-12-01	Code
14	DMAE (ViT-B/32)	53.4	No	Dual-Modal Attention-Enhanced Text-Video Retriev...	2023-09-20	Code
15	CAMoE	51	Yes	Improving Video-Text Retrieval by Multi-Stream C...	2021-09-09	Code
16	EMCL-Net++	50.6	No	Expectation-Maximization Contrastive Learning fo...	2022-11-21	Code
17	HiTeA	49.7	Yes	HiTeA: Hierarchical Temporal-Aware Video-Languag...	2022-12-30	-
18	DiffusionRet+QB-Norm	48.1	No	DiffusionRet: Generative Text-Video Retrieval wi...	2023-03-17	Code
19	Singularity	47.1	Yes	Revealing Single Frame Bias for Video-and-Langua...	2022-06-07	Code
20	CenterCLIP (ViT-B/16)	46.2	Yes	CenterCLIP: Token Clustering for Efficient Text-...	2022-05-02	Code
21	X-CLIP	46.2	No	X-CLIP: End-to-End Multi-grained Contrastive Lea...	2022-07-15	Code
22	DiffusionRet	45.8	No	DiffusionRet: Generative Text-Video Retrieval wi...	2023-03-17	Code
23	HBI	42.2	No	Video-Text as Game Players: Hierarchical Banzhaf...	2023-03-25	Code
24	EMCL-Net	41.2	No	Expectation-Maximization Contrastive Learning fo...	2022-11-21	Code
25	CLIP4Clip	40.5	No	CLIP4Clip: An Empirical Study of CLIP for End to...	2021-04-18	Code
26	TACo	30.4	Yes	TACo: Token-aware Cascade Contrastive Learning f...	2021-08-23	-
27	MMT-Pretrained	28.7	Yes	Multi-modal Transformer for Video Retrieval	2020-07-21	Code
28	HD-VILA	28.5	No	Advancing High-Resolution Video-Language Represe...	2021-11-19	Code
29	Ours	25.4	No	Video and Text Matching with Conditioned Embeddi...	2021-10-21	Code
30	MMT	22.7	No	Multi-modal Transformer for Video Retrieval	2020-07-21	Code
31	Collaborative Experts	20.5	No	Use What You Have: Video Retrieval Using Represe...	2019-07-31	Code

#1InternVideo2-6BSOTA
74.1
text-to-video R@1· Extra Data· 2024-03-22
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding Code
#2VASTSOTA
70.5
text-to-video R@1· Extra Data· 2023-05-29
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Code
#3VALORSOTA
70.1
text-to-video R@1· Extra Data· 2023-04-17
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Code
#4GRAM
69.9
text-to-video R@1· Extra Data· 2024-12-16
Gramian Multimodal Representation Learning and Alignment Code
#5COSA
67.3
text-to-video R@1· Extra Data· 2023-06-15
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model Code
#6UMT-L (ViT-L/16)SOTA
66.8
text-to-video R@1· Extra Data· 2023-03-28
Unmasked Teacher: Towards Training-Efficient Video Foundation Models Code
#7vid-TLDR (UMT-L)
66.7
text-to-video R@1· Extra Data· 2024-03-20
vid-TLDR: Training Free Token merging for Light-weight Video Transformer Code
#8InternVideoSOTA
62.2
text-to-video R@1· Extra Data· 2022-12-06
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Code
#9CLIP-ViPSOTA
61.4
text-to-video R@1· Extra Data· 2022-09-14
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Code
#10HunYuan_tvrSOTA
57.3
text-to-video R@1· Extra Data· 2022-04-07
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations
#11VindLU
55
text-to-video R@1· Extra Data· 2022-12-09
VindLU: A Recipe for Effective Video-and-Language Pretraining Code
#12TESTA (ViT-B/16)
54.8
text-to-video R@1· Extra Data· 2023-10-29
TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding Code
#13RTQ
53.5
text-to-video R@1· 2023-12-01
RTQ: Rethinking Video-language Understanding Based on Image-text Model Code
#14DMAE (ViT-B/32)
53.4
text-to-video R@1· 2023-09-20
Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning Code
#15CAMoESOTA
51
text-to-video R@1· Extra Data· 2021-09-09
Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Loss Code
#16EMCL-Net++
50.6
text-to-video R@1· 2022-11-21
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations Code
#17HiTeA
49.7
text-to-video R@1· Extra Data· 2022-12-30
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
#18DiffusionRet+QB-Norm
48.1
text-to-video R@1· 2023-03-17
DiffusionRet: Generative Text-Video Retrieval with Diffusion Model Code
#19Singularity
47.1
text-to-video R@1· Extra Data· 2022-06-07
Revealing Single Frame Bias for Video-and-Language Learning Code
#20CenterCLIP (ViT-B/16)
46.2
text-to-video R@1· Extra Data· 2022-05-02
CenterCLIP: Token Clustering for Efficient Text-Video Retrieval Code
#21X-CLIP
46.2
text-to-video R@1· 2022-07-15
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Code
#22DiffusionRet
45.8
text-to-video R@1· 2023-03-17
DiffusionRet: Generative Text-Video Retrieval with Diffusion Model Code
#23HBI
42.2
text-to-video R@1· 2023-03-25
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning Code
#24EMCL-Net
41.2
text-to-video R@1· 2022-11-21
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations Code
#25CLIP4ClipSOTA
40.5
text-to-video R@1· 2021-04-18
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Code
#26TACo
30.4
text-to-video R@1· Extra Data· 2021-08-23
TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment
#27MMT-PretrainedSOTA
28.7
text-to-video R@1· Extra Data· 2020-07-21
Multi-modal Transformer for Video Retrieval Code
#28HD-VILA
28.5
text-to-video R@1· 2021-11-19
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Code
#29Ours
25.4
text-to-video R@1· 2021-10-21
Video and Text Matching with Conditioned Embeddings Code
#30MMT
22.7
text-to-video R@1· 2020-07-21
Multi-modal Transformer for Video Retrieval Code
#31Collaborative ExpertsSOTA
20.5
text-to-video R@1· 2019-07-31
Use What You Have: Video Retrieval Using Representations From Collaborative Experts Code