Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/MDMMT-2

MDMMT-2

Reported on 50 benchmarks across 2 tasks · 1 paper · 24 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Vision50 results

VideoonYouCook2
text-to-video Mean Rank· uses extra data· 2022-03-14
12.7
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonYouCook2
text-to-video R@5· uses extra data· 2022-03-14
64
best: 74.3 (VAST)
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonLSMDC
text-to-video R@1· uses extra data· 2022-03-14
26.9
best: 46.4 (InternVideo2-6B)
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonLSMDC
text-to-video R@10· uses extra data· 2022-03-14
55.9
best: 92.8 (HunYuan_tvr (huge))
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonLSMDC
text-to-video R@5· uses extra data· 2022-03-14
46.7
best: 80.1 (HunYuan_tvr (huge))
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonTGIF
text-to-video Mean Rank· uses extra data· 2022-03-14
94.1
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonTGIF
text-to-video Median Rank· uses extra data· 2022-03-14
7
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonTGIF
text-to-video R@1· uses extra data· 2022-03-14
25.5
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonTGIF
text-to-video R@10· uses extra data· 2022-03-14
55.7
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonTGIF
text-to-video R@5· uses extra data· 2022-03-14
46.1
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSVD
text-to-video R@1· uses extra data· 2022-03-14
56.8
best: 61.4 (InternVideo2-6B)
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSVD
text-to-video R@10· uses extra data· 2022-03-14
89.2
best: 90.3 (HunYuan_tvr (huge))
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonYouCook2
text-to-video Mean Rank· uses extra data· 2022-03-14
12.7
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonYouCook2
text-to-video R@5· uses extra data· 2022-03-14
64
best: 74.3 (VAST)
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonLSMDC
text-to-video R@1· uses extra data· 2022-03-14
26.9
best: 46.4 (InternVideo2-6B)
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonLSMDC
text-to-video R@10· uses extra data· 2022-03-14
55.9
best: 92.8 (HunYuan_tvr (huge))
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonLSMDC
text-to-video R@5· uses extra data· 2022-03-14
46.7
best: 80.1 (HunYuan_tvr (huge))
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonTGIF
text-to-video Mean Rank· uses extra data· 2022-03-14
94.1
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonTGIF
text-to-video Median Rank· uses extra data· 2022-03-14
7
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonTGIF
text-to-video R@1· uses extra data· 2022-03-14
25.5
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonTGIF
text-to-video R@10· uses extra data· 2022-03-14
55.7
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonTGIF
text-to-video R@5· uses extra data· 2022-03-14
46.1
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSVD
text-to-video R@1· uses extra data· 2022-03-14
56.8
best: 61.4 (InternVideo2-6B)
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSVD
text-to-video R@10· uses extra data· 2022-03-14
89.2
best: 90.3 (HunYuan_tvr (huge))
SOTA
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonYouCook2
text-to-video Median Rank· uses extra data· 2022-03-14
3
best: 77 (Satar et al.)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonYouCook2
text-to-video R@1· uses extra data· 2022-03-14
32
best: 50.4 (VAST)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonYouCook2
text-to-video R@10· uses extra data· 2022-03-14
74.8
best: 80.8 (VAST)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSR-VTT
text-to-video Mean Rank· uses extra data· 2022-03-14
37.8
best: 213.8 (JEMC)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSR-VTT
text-to-video Median Rank· uses extra data· 2022-03-14
3
best: 55 (C+LSTM+SA+FC7)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSR-VTT
text-to-video R@1· uses extra data· 2022-03-14
33.7
best: 64 (GRAM)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSR-VTT
text-to-video R@10· uses extra data· 2022-03-14
70.8
best: 89.6 (VAST)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSR-VTT
text-to-video R@5· uses extra data· 2022-03-14
60.5
best: 84.3 (VAST)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonLSMDC
text-to-video Mean Rank· uses extra data· 2022-03-14
48
best: 58 (CLIP4Clip)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonLSMDC
text-to-video Median Rank· uses extra data· 2022-03-14
6.7
best: 56.5 (CLIP)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSVD
text-to-video Mean Rank· uses extra data· 2022-03-14
8.8
best: 23.1 (Collaborative Experts)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSVD
text-to-video Median Rank· uses extra data· 2022-03-14
1
best: 6 (SSML)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
VideoonMSVD
text-to-video R@5· uses extra data· 2022-03-14
83.1
best: 87.6 (CAMoE)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonYouCook2
text-to-video Median Rank· uses extra data· 2022-03-14
3
best: 77 (Satar et al.)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonYouCook2
text-to-video R@1· uses extra data· 2022-03-14
32
best: 50.4 (VAST)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonYouCook2
text-to-video R@10· uses extra data· 2022-03-14
74.8
best: 80.8 (VAST)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSR-VTT
text-to-video Mean Rank· uses extra data· 2022-03-14
37.8
best: 213.8 (JEMC)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSR-VTT
text-to-video Median Rank· uses extra data· 2022-03-14
3
best: 55 (C+LSTM+SA+FC7)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSR-VTT
text-to-video R@1· uses extra data· 2022-03-14
33.7
best: 64 (GRAM)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSR-VTT
text-to-video R@10· uses extra data· 2022-03-14
70.8
best: 89.6 (VAST)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSR-VTT
text-to-video R@5· uses extra data· 2022-03-14
60.5
best: 84.3 (VAST)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonLSMDC
text-to-video Mean Rank· uses extra data· 2022-03-14
48
best: 58 (CLIP4Clip)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonLSMDC
text-to-video Median Rank· uses extra data· 2022-03-14
6.7
best: 56.5 (CLIP)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSVD
text-to-video Mean Rank· uses extra data· 2022-03-14
8.8
best: 23.1 (Collaborative Experts)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSVD
text-to-video Median Rank· uses extra data· 2022-03-14
1
best: 6 (SSML)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086
Video RetrievalonMSVD
text-to-video R@5· uses extra data· 2022-03-14
83.1
best: 87.6 (CAMoE)
MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization arXiv:2203.07086