TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/Text-Video Embedding

Text-Video Embedding

Reported on 31 benchmarks across 6 tasks · 1 paper · 21 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Vision30 results

  • VideoonCrossTask
    Recall· 2019-06-07
    33.6
    best: 47.3 (VideoCLIP)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonYouCook2
    text-to-video Median Rank· 2019-06-07
    24
    best: 77 (Satar et al.)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonYouCook2
    text-to-video R@1· 2019-06-07
    8.2
    best: 50.4 (VAST)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonYouCook2
    text-to-video R@10· 2019-06-07
    35.3
    best: 80.8 (VAST)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonYouCook2
    text-to-video R@5· 2019-06-07
    24.5
    best: 74.3 (VAST)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonMSR-VTT
    text-to-video R@1· 2019-06-07
    14.9
    best: 64 (GRAM)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonMSR-VTT
    text-to-video R@10· 2019-06-07
    52.8
    best: 89.6 (VAST)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonMSR-VTT
    video-to-text R@5· 2019-06-07
    40.2
    best: 86.2 (CAMoE)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Temporal Action LocalizationonCrossTask
    Recall· 2019-06-07
    33.6
    best: 47.3 (VideoCLIP)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Action LocalizationonCrossTask
    Recall· 2019-06-07
    33.6
    best: 47.3 (VideoCLIP)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonYouCook2
    text-to-video Median Rank· 2019-06-07
    24
    best: 77 (Satar et al.)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonYouCook2
    text-to-video R@1· 2019-06-07
    8.2
    best: 50.4 (VAST)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonYouCook2
    text-to-video R@10· 2019-06-07
    35.3
    best: 80.8 (VAST)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonYouCook2
    text-to-video R@5· 2019-06-07
    24.5
    best: 74.3 (VAST)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonMSR-VTT
    text-to-video R@1· 2019-06-07
    14.9
    best: 64 (GRAM)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonMSR-VTT
    text-to-video R@10· 2019-06-07
    52.8
    best: 89.6 (VAST)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonMSR-VTT
    video-to-text R@5· 2019-06-07
    40.2
    best: 86.2 (CAMoE)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Long Video Retrieval (Background Removed)onYouCook2
    Cap. Avg. R@1· 2019-06-07
    46.6
    best: 75.5 (Norton)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Long Video Retrieval (Background Removed)onYouCook2
    Cap. Avg. R@10· 2019-06-07
    83.7
    best: 97.9 (VideoCLIP)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Long Video Retrieval (Background Removed)onYouCook2
    Cap. Avg. R@5· 2019-06-07
    74.3
    best: 95 (Norton)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonMSR-VTT
    text-to-video Median Rank· 2019-06-07
    9
    best: 55 (C+LSTM+SA+FC7)
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonLSMDC
    text-to-video Median Rank· 2019-06-07
    40
    best: 56.5 (CLIP)
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonLSMDC
    text-to-video R@1· 2019-06-07
    7.2
    best: 46.4 (InternVideo2-6B)
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonLSMDC
    text-to-video R@10· 2019-06-07
    27.9
    best: 92.8 (HunYuan_tvr (huge))
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • VideoonLSMDC
    text-to-video R@5· 2019-06-07
    19.6
    best: 80.1 (HunYuan_tvr (huge))
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonMSR-VTT
    text-to-video Median Rank· 2019-06-07
    9
    best: 55 (C+LSTM+SA+FC7)
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonLSMDC
    text-to-video Median Rank· 2019-06-07
    40
    best: 56.5 (CLIP)
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonLSMDC
    text-to-video R@1· 2019-06-07
    7.2
    best: 46.4 (InternVideo2-6B)
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonLSMDC
    text-to-video R@10· 2019-06-07
    27.9
    best: 92.8 (HunYuan_tvr (huge))
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327
  • Video RetrievalonLSMDC
    text-to-video R@5· 2019-06-07
    19.6
    best: 80.1 (HunYuan_tvr (huge))
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327

Methodology1 result

  • Zero-Shot LearningonCrossTask
    Recall· 2019-06-07
    33.6
    best: 47.3 (VideoCLIP)
    SOTA
    HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video ClipsarXiv:1906.03327