Zero-Shot Video Retrieval on DiDeMo

Metric: text-to-video R@5 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	text-to-video R@5▼	Extra Data	Paper	Date↕	Code
1	InternVideo2-6B	80	Yes	InternVideo2: Scaling Foundation Models for Mult...	2024-03-22	Code
2	InternVideo2-1B	80	Yes	InternVideo2: Scaling Foundation Models for Mult...	2024-03-22	Code
3	VAST	74.3	Yes	VAST: A Vision-Audio-Subtitle-Text Omni-Modality...	2023-05-29	Code
4	vid-TLDR (UMT-L)	74	Yes	vid-TLDR: Training Free Token merging for Light-...	2024-03-20	Code
5	UMT-L (ViT-L/16)	72.9	Yes	Unmasked Teacher: Towards Training-Efficient Vid...	2023-03-28	Code
6	mPLUG-2	71.1	No	mPLUG-2: A Modularized Multi-modal Foundation Mo...	2023-02-01	Code
7	HiTeA-17M	69.3	Yes	HiTeA: Hierarchical Temporal-Aware Video-Languag...	2022-12-30	-
8	LanguageBind(ViT-H/14)	66.1	Yes	LanguageBind: Extending Video-Language Pretraini...	2023-10-03	Code
9	LanguageBind(ViT-L/14)	65.5	Yes	LanguageBind: Extending Video-Language Pretraini...	2023-10-03	Code
10	BT-Adapter	61.9	No	BT-Adapter: Video Conversation is Feasible Witho...	2023-09-27	Code
11	Singularity-17M	61.7	Yes	Revealing Single Frame Bias for Video-and-Langua...	2022-06-07	Code
12	Singularity-5M	61.1	Yes	Revealing Single Frame Bias for Video-and-Langua...	2022-06-07	Code
13	HiTeA-5M	60.1	Yes	HiTeA: Hierarchical Temporal-Aware Video-Languag...	2022-12-30	-
14	OmniVL	58.7	Yes	OmniVL:One Foundation Model for Image-Language a...	2022-09-15	-
15	InternVideo	57.6	Yes	InternVideo: General Video Foundation Models via...	2022-12-06	Code
16	Clover	55.2	Yes	Clover: Towards A Unified Video-Language Alignme...	2022-07-16	Code
17	Y. Ge et. al.	50.6	No	Bridging Video-text Retrieval with Multiple Choi...	2022-01-13	Code
18	OA-Trans	50.4	No	Object-aware Video-language Pre-training for Ret...	2021-12-01	Code
19	MILES	50.3	No	MILES: Visual BERT Pre-training with Injected La...	2022-04-26	Code
20	VIOLET	49.8	No	VIOLET : End-to-End Video-Language Transformers ...	2021-11-24	Code
21	ALPRO	47.3	No	Align and Prompt: Video-and-Language Pre-trainin...	2021-12-17	Code
22	VideoCLIP	46.9	No	VideoCLIP: Contrastive Pre-training for Zero-sho...	2021-09-28	Code
23	M. Bain et. al.	46.4	No	Frozen in Time: A Joint Video and Image Encoder ...	2021-04-01	Code
24	FROZEN	46	Yes	Frozen in Time: A Joint Video and Image Encoder ...	2021-04-01	Code
25	LaT	45.9	No	LaT: Latent Translation with Cycle-Consistency f...	2022-07-11	-

#1InternVideo2-6BSOTA
80
text-to-video R@5· Extra Data· 2024-03-22
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding Code
#2InternVideo2-1B
80
text-to-video R@5· Extra Data· 2024-03-22
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding Code
#3VASTSOTA
74.3
text-to-video R@5· Extra Data· 2023-05-29
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Code
#4vid-TLDR (UMT-L)
74
text-to-video R@5· Extra Data· 2024-03-20
vid-TLDR: Training Free Token merging for Light-weight Video Transformer Code
#5UMT-L (ViT-L/16)SOTA
72.9
text-to-video R@5· Extra Data· 2023-03-28
Unmasked Teacher: Towards Training-Efficient Video Foundation Models Code
#6mPLUG-2SOTA
71.1
text-to-video R@5· 2023-02-01
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video Code
#7HiTeA-17MSOTA
69.3
text-to-video R@5· Extra Data· 2022-12-30
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
#8LanguageBind(ViT-H/14)
66.1
text-to-video R@5· Extra Data· 2023-10-03
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment Code
#9LanguageBind(ViT-L/14)
65.5
text-to-video R@5· Extra Data· 2023-10-03
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment Code
#10BT-Adapter
61.9
text-to-video R@5· 2023-09-27
BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning Code
#11Singularity-17MSOTA
61.7
text-to-video R@5· Extra Data· 2022-06-07
Revealing Single Frame Bias for Video-and-Language Learning Code
#12Singularity-5M
61.1
text-to-video R@5· Extra Data· 2022-06-07
Revealing Single Frame Bias for Video-and-Language Learning Code
#13HiTeA-5M
60.1
text-to-video R@5· Extra Data· 2022-12-30
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
#14OmniVL
58.7
text-to-video R@5· Extra Data· 2022-09-15
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks
#15InternVideo
57.6
text-to-video R@5· Extra Data· 2022-12-06
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Code
#16Clover
55.2
text-to-video R@5· Extra Data· 2022-07-16
Clover: Towards A Unified Video-Language Alignment and Fusion Model Code
#17Y. Ge et. al.SOTA
50.6
text-to-video R@5· 2022-01-13
Bridging Video-text Retrieval with Multiple Choice Questions Code
#18OA-TransSOTA
50.4
text-to-video R@5· 2021-12-01
Object-aware Video-language Pre-training for Retrieval Code
#19MILES
50.3
text-to-video R@5· 2022-04-26
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval Code
#20VIOLETSOTA
49.8
text-to-video R@5· 2021-11-24
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Code
#21ALPRO
47.3
text-to-video R@5· 2021-12-17
Align and Prompt: Video-and-Language Pre-training with Entity Prompts Code
#22VideoCLIPSOTA
46.9
text-to-video R@5· 2021-09-28
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Code
#23M. Bain et. al.SOTA
46.4
text-to-video R@5· 2021-04-01
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Code
#24FROZEN
46
text-to-video R@5· Extra Data· 2021-04-01
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Code
#25LaT
45.9
text-to-video R@5· 2022-07-11
LaT: Latent Translation with Cycle-Consistency for Video-Text Retrieval