Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/Clover

Clover

Reported on 41 benchmarks across 5 tasks · 1 paper · 5 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Vision36 results

Zero-Shot Video RetrievalonLSMDC
text-to-video R@10· uses extra data· 2022-07-16
38.2
best: 62.2 (InternVideo2-6B)
SOTA
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonLSMDC
text-to-video R@5· uses extra data· 2022-07-16
29.2
best: 55.9 (InternVideo2-6B)
SOTA
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonMSR-VTT-1kA
text-to-video Median Rank· 2022-07-16
2
best: 13 (JSFusion)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonMSR-VTT-1kA
text-to-video R@1· 2022-07-16
40.5
best: 62.9 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonMSR-VTT-1kA
text-to-video R@10· 2022-07-16
79.4
best: 90.8 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonMSR-VTT-1kA
text-to-video R@5· 2022-07-16
69.8
best: 84.5 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonDiDeMo
text-to-video Median Rank· 2022-07-16
1
best: 8.3 (Collaborative Experts)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonDiDeMo
text-to-video R@1· 2022-07-16
50.1
best: 74.2 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonDiDeMo
text-to-video R@10· 2022-07-16
85.6
best: 94.2 (vid-TLDR (UMT-L))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonDiDeMo
text-to-video R@5· 2022-07-16
76.7
best: 91.2 (vid-TLDR (UMT-L))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonLSMDC
text-to-video Median Rank· 2022-07-16
8
best: 56.5 (CLIP)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonLSMDC
text-to-video R@1· 2022-07-16
24.8
best: 46.4 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonLSMDC
text-to-video R@10· 2022-07-16
54.5
best: 92.8 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
VideoonLSMDC
text-to-video R@5· 2022-07-16
44
best: 80.1 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonMSR-VTT-1kA
text-to-video Median Rank· 2022-07-16
2
best: 13 (JSFusion)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonMSR-VTT-1kA
text-to-video R@1· 2022-07-16
40.5
best: 62.9 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonMSR-VTT-1kA
text-to-video R@10· 2022-07-16
79.4
best: 90.8 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonMSR-VTT-1kA
text-to-video R@5· 2022-07-16
69.8
best: 84.5 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonDiDeMo
text-to-video Median Rank· 2022-07-16
1
best: 8.3 (Collaborative Experts)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonDiDeMo
text-to-video R@1· 2022-07-16
50.1
best: 74.2 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonDiDeMo
text-to-video R@10· 2022-07-16
85.6
best: 94.2 (vid-TLDR (UMT-L))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonDiDeMo
text-to-video R@5· 2022-07-16
76.7
best: 91.2 (vid-TLDR (UMT-L))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonLSMDC
text-to-video Median Rank· 2022-07-16
8
best: 56.5 (CLIP)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonLSMDC
text-to-video R@1· 2022-07-16
24.8
best: 46.4 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonLSMDC
text-to-video R@10· 2022-07-16
54.5
best: 92.8 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video RetrievalonLSMDC
text-to-video R@5· 2022-07-16
44
best: 80.1 (HunYuan_tvr (huge))
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonMSR-VTT
text-to-video Median Rank· 2022-07-16
6
best: 66 (MMT)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonMSR-VTT
text-to-video R@1· 2022-07-16
26.4
best: 55.9 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonMSR-VTT
text-to-video R@10· 2022-07-16
60
best: 85.1 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonMSR-VTT
text-to-video R@5· 2022-07-16
49.5
best: 78.3 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonDiDeMo
text-to-video Median Rank· uses extra data· 2022-07-16
4
best: 7 (LaT)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonDiDeMo
text-to-video R@1· uses extra data· 2022-07-16
29.5
best: 57.9 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonDiDeMo
text-to-video R@10· uses extra data· 2022-07-16
66.3
best: 85.1 (InternVideo2-1B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonDiDeMo
text-to-video R@5· uses extra data· 2022-07-16
55.2
best: 80 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonLSMDC
text-to-video Median Rank· uses extra data· 2022-07-16
24
best: 50.7 (MILES)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Zero-Shot Video RetrievalonLSMDC
text-to-video R@1· uses extra data· 2022-07-16
14.7
best: 33.8 (InternVideo2-6B)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885

Reasoning3 results

Video Question AnsweringonLSMDC-FiB
Accuracy· 2022-07-16
54.1
SOTA
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video Question AnsweringonLSMDC-MC
Accuracy· 2022-07-16
83.7
best: 84.4 (VIOLETv2)
SOTA
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Video Question AnsweringonMSRVTT-MC
Accuracy· 2022-07-16
95.2
best: 97.6 (VIOLETv2)
SOTA
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885

Natural Language Processing2 results

Visual Question Answering (VQA)onMSRVTT-QA
Accuracy· uses extra data· 2022-07-16
0.441
best: 0.496 (VLAB)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885
Visual Question Answering (VQA)onMSVD-QA
Accuracy· uses extra data· 2022-07-16
0.524
best: 0.61 (VLAB)
Clover: Towards A Unified Video-Language Alignment and Fusion Model arXiv:2207.07885