Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/X-CLIP

X-CLIP

Reported on 64 benchmarks across 3 tasks · 2 papers · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Vision64 results

VideoonActivityNet
video-to-text Mean Rank· 2022-07-15
6.4
best: 6.7 (DiffusionRet+QB-Norm)
SOTA
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonActivityNet
video-to-text Mean Rank· 2022-07-15
6.4
best: 6.7 (DiffusionRet+QB-Norm)
SOTA
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Zero-Shot Action RecognitiononUCF101
Top-1 Accuracy· 2022-08-04
72
best: 92.8 (OTI(ViT-L/14))
Expanding Language-Image Pretrained Models for General Video Recognition arXiv:2208.02816
Zero-Shot Action RecognitiononKinetics
Top-1 Accuracy· 2022-08-04
65.2
best: 78.1 (TC-CLIP)
Expanding Language-Image Pretrained Models for General Video Recognition arXiv:2208.02816
Zero-Shot Action RecognitiononKinetics
Top-5 Accuracy· 2022-08-04
86.1
best: 95.7 (TC-CLIP)
Expanding Language-Image Pretrained Models for General Video Recognition arXiv:2208.02816
Zero-Shot Action RecognitiononHMDB51
Top-1 Accuracy· 2022-08-04
44.6
best: 64.7 (MOV (ViT-L/14))
Expanding Language-Image Pretrained Models for General Video Recognition arXiv:2208.02816
VideoonMSR-VTT-1kA
text-to-video Mean Rank· 2022-07-15
12.2
best: 28.2 (Collaborative Experts)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSR-VTT-1kA
text-to-video Median Rank· 2022-07-15
2
best: 13 (JSFusion)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSR-VTT-1kA
text-to-video R@1· 2022-07-15
49.3
best: 62.9 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSR-VTT-1kA
text-to-video R@10· 2022-07-15
84.8
best: 90.8 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSR-VTT-1kA
text-to-video R@5· 2022-07-15
75.8
best: 84.5 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSR-VTT-1kA
video-to-text Mean Rank· 2022-07-15
8.1
best: 10.2 (CenterCLIP (ViT-B/16))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSR-VTT-1kA
video-to-text Median Rank· 2022-07-15
2
best: 5 (CLIP)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSR-VTT-1kA
video-to-text R@1· 2022-07-15
48.9
best: 64.8 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSR-VTT-1kA
video-to-text R@10· 2022-07-15
84.5
best: 91.1 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSR-VTT-1kA
video-to-text R@5· 2022-07-15
76.8
best: 783 (PIDRo)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonActivityNet
text-to-video Mean Rank· 2022-07-15
6.8
best: 23.1 (Collaborative Experts)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonActivityNet
text-to-video R@1· 2022-07-15
46.2
best: 74.1 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonActivityNet
text-to-video R@5· 2022-07-15
75.5
best: 90.9 (VAST)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonActivityNet
video-to-text R@1· 2022-07-15
46.4
best: 69.7 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonActivityNet
video-to-text R@5· 2022-07-15
75.9
best: 89.1 (UMT-L (ViT-L/16))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonDiDeMo
text-to-video Mean Rank· 2022-07-15
12.6
best: 43.7 (Collaborative Experts)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonDiDeMo
text-to-video R@1· 2022-07-15
47.8
best: 74.2 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonDiDeMo
text-to-video R@5· 2022-07-15
79.3
best: 91.2 (vid-TLDR (UMT-L))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonDiDeMo
video-to-text Mean Rank· 2022-07-15
10.5
best: 39.6 (PO Loss)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonDiDeMo
video-to-text R@1· 2022-07-15
47.8
best: 71.9 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonDiDeMo
video-to-text R@10· 2022-07-15
76.8
best: 93.8 (vid-TLDR (UMT-L))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonLSMDC
text-to-video R@1· 2022-07-15
26.1
best: 46.4 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonLSMDC
video-to-text R@1· 2022-07-15
26.9
best: 46.7 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSVD
text-to-video Mean Rank· 2022-07-15
8.4
best: 23.1 (Collaborative Experts)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSVD
text-to-video R@1· 2022-07-15
50.4
best: 61.4 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSVD
text-to-video R@5· 2022-07-15
80.6
best: 87.6 (CAMoE)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSVD
video-to-text Mean Rank· 2022-07-15
4.2
best: 7.6 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSVD
video-to-text R@1· 2022-07-15
66.8
best: 85.2 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
VideoonMSVD
video-to-text R@10· 2022-07-15
90.4
best: 97.1 (PAU)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
text-to-video Mean Rank· 2022-07-15
12.2
best: 28.2 (Collaborative Experts)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
text-to-video Median Rank· 2022-07-15
2
best: 13 (JSFusion)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
text-to-video R@1· 2022-07-15
49.3
best: 62.9 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
text-to-video R@10· 2022-07-15
84.8
best: 90.8 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
text-to-video R@5· 2022-07-15
75.8
best: 84.5 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
video-to-text Mean Rank· 2022-07-15
8.1
best: 10.2 (CenterCLIP (ViT-B/16))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
video-to-text Median Rank· 2022-07-15
2
best: 5 (CLIP)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
video-to-text R@1· 2022-07-15
48.9
best: 64.8 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
video-to-text R@10· 2022-07-15
84.5
best: 91.1 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSR-VTT-1kA
video-to-text R@5· 2022-07-15
76.8
best: 783 (PIDRo)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonActivityNet
text-to-video Mean Rank· 2022-07-15
6.8
best: 23.1 (Collaborative Experts)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonActivityNet
text-to-video R@1· 2022-07-15
46.2
best: 74.1 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonActivityNet
text-to-video R@5· 2022-07-15
75.5
best: 90.9 (VAST)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonActivityNet
video-to-text R@1· 2022-07-15
46.4
best: 69.7 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonActivityNet
video-to-text R@5· 2022-07-15
75.9
best: 89.1 (UMT-L (ViT-L/16))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonDiDeMo
text-to-video Mean Rank· 2022-07-15
12.6
best: 43.7 (Collaborative Experts)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonDiDeMo
text-to-video R@1· 2022-07-15
47.8
best: 74.2 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonDiDeMo
text-to-video R@5· 2022-07-15
79.3
best: 91.2 (vid-TLDR (UMT-L))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonDiDeMo
video-to-text Mean Rank· 2022-07-15
10.5
best: 39.6 (PO Loss)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonDiDeMo
video-to-text R@1· 2022-07-15
47.8
best: 71.9 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonDiDeMo
video-to-text R@10· 2022-07-15
76.8
best: 93.8 (vid-TLDR (UMT-L))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonLSMDC
text-to-video R@1· 2022-07-15
26.1
best: 46.4 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonLSMDC
video-to-text R@1· 2022-07-15
26.9
best: 46.7 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSVD
text-to-video Mean Rank· 2022-07-15
8.4
best: 23.1 (Collaborative Experts)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSVD
text-to-video R@1· 2022-07-15
50.4
best: 61.4 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSVD
text-to-video R@5· 2022-07-15
80.6
best: 87.6 (CAMoE)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSVD
video-to-text Mean Rank· 2022-07-15
4.2
best: 7.6 (HunYuan_tvr (huge))
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSVD
video-to-text R@1· 2022-07-15
66.8
best: 85.2 (InternVideo2-6B)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285
Video RetrievalonMSVD
video-to-text R@10· 2022-07-15
90.4
best: 97.1 (PAU)
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval arXiv:2207.07285