Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/VLM

VLM

Reported on 27 benchmarks across 7 tasks · 1 paper · 4 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Vision26 results

VideoonCrossTask
Recall· uses extra data· 2021-05-20
46.5
best: 47.3 (VideoCLIP)
SOTA
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Temporal Action LocalizationonCrossTask
Recall· uses extra data· 2021-05-20
46.5
best: 47.3 (VideoCLIP)
SOTA
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Action LocalizationonCrossTask
Recall· uses extra data· 2021-05-20
46.5
best: 47.3 (VideoCLIP)
SOTA
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
VideoonMSR-VTT-1kA
text-to-video Median Rank· uses extra data· 2021-05-20
4
best: 13 (JSFusion)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
VideoonMSR-VTT-1kA
text-to-video R@1· uses extra data· 2021-05-20
28.1
best: 62.9 (HunYuan_tvr (huge))
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
VideoonMSR-VTT-1kA
text-to-video R@10· uses extra data· 2021-05-20
67.4
best: 90.8 (HunYuan_tvr (huge))
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
VideoonMSR-VTT-1kA
text-to-video R@5· uses extra data· 2021-05-20
55.5
best: 84.5 (HunYuan_tvr (huge))
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
VideoonYouCook2
text-to-video Median Rank· uses extra data· 2021-05-20
4
best: 77 (Satar et al.)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
VideoonYouCook2
text-to-video R@1· uses extra data· 2021-05-20
27.05
best: 50.4 (VAST)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
VideoonYouCook2
text-to-video R@10· uses extra data· 2021-05-20
69.38
best: 80.8 (VAST)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
VideoonYouCook2
text-to-video R@5· uses extra data· 2021-05-20
56.88
best: 74.3 (VAST)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Action LocalizationonCOIN
Frame accuracy· uses extra data· 2021-05-20
68.4
best: 72.8 (UnLoc-L)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video CaptioningonYouCook2
BLEU-3· uses extra data· 2021-05-20
17.78
best: 24.12 (UniVL + MELTR)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video CaptioningonYouCook2
BLEU-4· uses extra data· 2021-05-20
12.27
best: 18.2 (VAST)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video CaptioningonYouCook2
CIDEr· uses extra data· 2021-05-20
1.3869
best: 116.4 (HowToCaption)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video CaptioningonYouCook2
METEOR· uses extra data· 2021-05-20
18.22
best: 22.56 (UniVL + MELTR)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video CaptioningonYouCook2
ROUGE-L· uses extra data· 2021-05-20
41.51
best: 47.04 (UniVL + MELTR)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video RetrievalonMSR-VTT-1kA
text-to-video Median Rank· uses extra data· 2021-05-20
4
best: 13 (JSFusion)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video RetrievalonMSR-VTT-1kA
text-to-video R@1· uses extra data· 2021-05-20
28.1
best: 62.9 (HunYuan_tvr (huge))
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video RetrievalonMSR-VTT-1kA
text-to-video R@10· uses extra data· 2021-05-20
67.4
best: 90.8 (HunYuan_tvr (huge))
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video RetrievalonMSR-VTT-1kA
text-to-video R@5· uses extra data· 2021-05-20
55.5
best: 84.5 (HunYuan_tvr (huge))
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video RetrievalonYouCook2
text-to-video Median Rank· uses extra data· 2021-05-20
4
best: 77 (Satar et al.)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video RetrievalonYouCook2
text-to-video R@1· uses extra data· 2021-05-20
27.05
best: 50.4 (VAST)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video RetrievalonYouCook2
text-to-video R@10· uses extra data· 2021-05-20
69.38
best: 80.8 (VAST)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Video RetrievalonYouCook2
text-to-video R@5· uses extra data· 2021-05-20
56.88
best: 74.3 (VAST)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996
Action SegmentationonCOIN
Frame accuracy· uses extra data· 2021-05-20
68.4
best: 72.8 (UnLoc-L)
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996

Methodology1 result

Zero-Shot LearningonCrossTask
Recall· uses extra data· 2021-05-20
46.5
best: 47.3 (VideoCLIP)
SOTA
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding arXiv:2105.09996