Video Retrieval on DiDeMo

Metric: text-to-video R@1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	text-to-video R@1▼	Extra Data	Paper	Date↕	Code
1	InternVideo2-6B	74.2	Yes	InternVideo2: Scaling Foundation Models for Mult...	2024-03-22	Code
2	vid-TLDR (UMT-L)	72.3	Yes	vid-TLDR: Training Free Token merging for Light-...	2024-03-20	Code
3	VAST	72	Yes	VAST: A Vision-Audio-Subtitle-Text Omni-Modality...	2023-05-29	Code
4	COSA	70.5	Yes	COSA: Concatenated Sample Pretrained Vision-Lang...	2023-06-15	Code
5	UMT-L (ViT-L/16)	70.4	Yes	Unmasked Teacher: Towards Training-Efficient Vid...	2023-03-28	Code
6	GRAM	67.3	Yes	Gramian Multimodal Representation Learning and A...	2024-12-16	Code
7	VALOR	61.5	Yes	VALOR: Vision-Audio-Language Omni-Perception Pre...	2023-04-17	Code
8	TESTA (ViT-B/16)	61.2	Yes	TESTA: Temporal-Spatial Token Aggregation for Lo...	2023-10-29	Code
9	VindLU	61.2	Yes	VindLU: A Recipe for Effective Video-and-Languag...	2022-12-09	Code
10	InternVideo	57.9	Yes	InternVideo: General Video Foundation Models via...	2022-12-06	Code
11	RTQ	57.6	No	RTQ: Rethinking Video-language Understanding Bas...	2023-12-01	Code
12	VLAB	56.8	Yes	VLAB: Enhancing Video Language Pre-training by F...	2023-05-22	-
13	HiTeA	56.5	Yes	HiTeA: Hierarchical Temporal-Aware Video-Languag...	2022-12-30	-
14	MuLTI	56.5	Yes	MuLTI: Efficient Video-and-Language Understandin...	2023-03-10	-
15	mPLUG-2	56.4	Yes	mPLUG-2: A Modularized Multi-modal Foundation Mo...	2023-02-01	Code
16	CLIP-ViP	55.3	Yes	CLIP-ViP: Adapting Pre-trained Image-Text Model ...	2022-09-14	Code
17	STAN	54.6	Yes	Revisiting Temporal Modeling for CLIP-based Imag...	2023-01-26	Code
18	Singularity	53.9	Yes	Revealing Single Frame Bias for Video-and-Langua...	2022-06-07	Code
19	DMAE (ViT-B/32)	52.7	No	Dual-Modal Attention-Enhanced Text-Video Retriev...	2023-09-20	Code
20	HunYuan_tvr (huge)	52.7	Yes	Tencent Text-Video Retrieval: Hierarchical Cross...	2022-04-07	-
21	OmniVL	52.4	Yes	OmniVL:One Foundation Model for Image-Language a...	2022-09-15	-
22	HunYuan_tvr	52.1	Yes	Tencent Text-Video Retrieval: Hierarchical Cross...	2022-04-07	-
23	Cap4Video	52	No	Cap4Video: What Can Auxiliary Captions Do for Te...	2022-12-31	Code
24	Clover	50.1	No	Clover: Towards A Unified Video-Language Alignme...	2022-07-16	Code
25	DRL	49	Yes	Disentangled Representation Learning for Text-Vi...	2022-03-14	Code
26	DiffusionRet+QB-Norm	48.9	No	DiffusionRet: Generative Text-Video Retrieval wi...	2023-03-17	Code
27	PAU	48.6	No	Prototype-based Aleatoric Uncertainty Quantifica...	2023-09-29	Code
28	VIOLETv2	47.9	No	An Empirical Study of End-to-End Video-Language ...	2022-09-04	Code
29	X-CLIP	47.8	No	X-CLIP: End-to-End Multi-grained Contrastive Lea...	2022-07-15	Code
30	HBI	46.9	No	Video-Text as Game Players: Hierarchical Banzhaf...	2023-03-25	Code
31	DiffusionRet	46.7	No	DiffusionRet: Generative Text-Video Retrieval wi...	2023-03-17	Code
32	CAMoE	43.8	Yes	Improving Video-Text Retrieval by Multi-Stream C...	2021-09-09	Code
33	QB-Norm+CLIP4Clip	43.5	Yes	Cross Modal Retrieval with Querybank Normalisation	2021-12-23	Code
34	CLIP4Clip	43.4	Yes	CLIP4Clip: An Empirical Study of CLIP for End to...	2021-04-18	Code
35	ALPRO	35.9	Yes	Align and Prompt: Video-and-Language Pre-trainin...	2021-12-17	Code
36	FROZEN	31	Yes	Frozen in Time: A Joint Video and Image Encoder ...	2021-04-01	Code
37	HD-VILA	28.8	No	Advancing High-Resolution Video-Language Represe...	2021-11-19	Code
38	PO Loss	16.3	No	Rudder: A Cross Lingual Video and Text Retrieval...	2021-03-09	Code
39	Collaborative Experts	16.1	No	Use What You Have: Video Retrieval Using Represe...	2019-07-31	Code

#1InternVideo2-6BSOTA
74.2
text-to-video R@1· Extra Data· 2024-03-22
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding Code
#2vid-TLDR (UMT-L)SOTA
72.3
text-to-video R@1· Extra Data· 2024-03-20
vid-TLDR: Training Free Token merging for Light-weight Video Transformer Code
#3VASTSOTA
72
text-to-video R@1· Extra Data· 2023-05-29
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Code
#4COSA
70.5
text-to-video R@1· Extra Data· 2023-06-15
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model Code
#5UMT-L (ViT-L/16)SOTA
70.4
text-to-video R@1· Extra Data· 2023-03-28
Unmasked Teacher: Towards Training-Efficient Video Foundation Models Code
#6GRAM
67.3
text-to-video R@1· Extra Data· 2024-12-16
Gramian Multimodal Representation Learning and Alignment Code
#7VALOR
61.5
text-to-video R@1· Extra Data· 2023-04-17
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Code
#8TESTA (ViT-B/16)
61.2
text-to-video R@1· Extra Data· 2023-10-29
TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding Code
#9VindLUSOTA
61.2
text-to-video R@1· Extra Data· 2022-12-09
VindLU: A Recipe for Effective Video-and-Language Pretraining Code
#10InternVideoSOTA
57.9
text-to-video R@1· Extra Data· 2022-12-06
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Code
#11RTQ
57.6
text-to-video R@1· 2023-12-01
RTQ: Rethinking Video-language Understanding Based on Image-text Model Code
#12VLAB
56.8
text-to-video R@1· Extra Data· 2023-05-22
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending
#13HiTeA
56.5
text-to-video R@1· Extra Data· 2022-12-30
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
#14MuLTI
56.5
text-to-video R@1· Extra Data· 2023-03-10
MuLTI: Efficient Video-and-Language Understanding with Text-Guided MultiWay-Sampler and Multiple Choice Modeling
#15mPLUG-2
56.4
text-to-video R@1· Extra Data· 2023-02-01
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video Code
#16CLIP-ViPSOTA
55.3
text-to-video R@1· Extra Data· 2022-09-14
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Code
#17STAN
54.6
text-to-video R@1· Extra Data· 2023-01-26
Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferring Code
#18SingularitySOTA
53.9
text-to-video R@1· Extra Data· 2022-06-07
Revealing Single Frame Bias for Video-and-Language Learning Code
#19DMAE (ViT-B/32)
52.7
text-to-video R@1· 2023-09-20
Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning Code
#20HunYuan_tvr (huge)SOTA
52.7
text-to-video R@1· Extra Data· 2022-04-07
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations
#21OmniVL
52.4
text-to-video R@1· Extra Data· 2022-09-15
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks
#22HunYuan_tvr
52.1
text-to-video R@1· Extra Data· 2022-04-07
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations
#23Cap4Video
52
text-to-video R@1· 2022-12-31
Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?Code
#24Clover
50.1
text-to-video R@1· 2022-07-16
Clover: Towards A Unified Video-Language Alignment and Fusion Model Code
#25DRLSOTA
49
text-to-video R@1· Extra Data· 2022-03-14
Disentangled Representation Learning for Text-Video Retrieval Code
#26DiffusionRet+QB-Norm
48.9
text-to-video R@1· 2023-03-17
DiffusionRet: Generative Text-Video Retrieval with Diffusion Model Code
#27PAU
48.6
text-to-video R@1· 2023-09-29
Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval Code
#28VIOLETv2
47.9
text-to-video R@1· 2022-09-04
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling Code
#29X-CLIP
47.8
text-to-video R@1· 2022-07-15
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Code
#30HBI
46.9
text-to-video R@1· 2023-03-25
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning Code
#31DiffusionRet
46.7
text-to-video R@1· 2023-03-17
DiffusionRet: Generative Text-Video Retrieval with Diffusion Model Code
#32CAMoESOTA
43.8
text-to-video R@1· Extra Data· 2021-09-09
Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Loss Code
#33QB-Norm+CLIP4Clip
43.5
text-to-video R@1· Extra Data· 2021-12-23
Cross Modal Retrieval with Querybank Normalisation Code
#34CLIP4ClipSOTA
43.4
text-to-video R@1· Extra Data· 2021-04-18
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Code
#35ALPRO
35.9
text-to-video R@1· Extra Data· 2021-12-17
Align and Prompt: Video-and-Language Pre-training with Entity Prompts Code
#36FROZENSOTA
31
text-to-video R@1· Extra Data· 2021-04-01
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Code
#37HD-VILA
28.8
text-to-video R@1· 2021-11-19
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Code
#38PO LossSOTA
16.3
text-to-video R@1· 2021-03-09
Rudder: A Cross Lingual Video and Text Retrieval Dataset Code
#39Collaborative ExpertsSOTA
16.1
text-to-video R@1· 2019-07-31
Use What You Have: Video Retrieval Using Representations From Collaborative Experts Code