TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/VIOLETv2

VIOLETv2

Reported on 24 benchmarks across 5 tasks · 1 paper · 3 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Vision20 results

  • Video CaptioningonMSVD
    CIDEr· 2022-09-04
    139.2
    best: 195.6 (MaMMUT)
    SOTA
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • VideoonDiDeMo
    text-to-video R@1· 2022-09-04
    47.9
    best: 74.2 (InternVideo2-6B)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • VideoonDiDeMo
    text-to-video R@10· 2022-09-04
    84.1
    best: 94.2 (vid-TLDR (UMT-L))
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • VideoonDiDeMo
    text-to-video R@5· 2022-09-04
    76.5
    best: 91.2 (vid-TLDR (UMT-L))
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • VideoonMSR-VTT
    text-to-video R@1· uses extra data· 2022-09-04
    37.2
    best: 64 (GRAM)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • VideoonMSR-VTT
    text-to-video R@10· uses extra data· 2022-09-04
    75.8
    best: 89.6 (VAST)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • VideoonMSR-VTT
    text-to-video R@5· uses extra data· 2022-09-04
    64.8
    best: 84.3 (VAST)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • VideoonLSMDC
    text-to-video R@1· 2022-09-04
    24
    best: 46.4 (InternVideo2-6B)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • VideoonLSMDC
    text-to-video R@10· 2022-09-04
    54.1
    best: 92.8 (HunYuan_tvr (huge))
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • VideoonLSMDC
    text-to-video R@5· 2022-09-04
    43.5
    best: 80.1 (HunYuan_tvr (huge))
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video CaptioningonMSR-VTT
    CIDEr· 2022-09-04
    58
    best: 80 (mPLUG-2)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video RetrievalonDiDeMo
    text-to-video R@1· 2022-09-04
    47.9
    best: 74.2 (InternVideo2-6B)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video RetrievalonDiDeMo
    text-to-video R@10· 2022-09-04
    84.1
    best: 94.2 (vid-TLDR (UMT-L))
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video RetrievalonDiDeMo
    text-to-video R@5· 2022-09-04
    76.5
    best: 91.2 (vid-TLDR (UMT-L))
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video RetrievalonMSR-VTT
    text-to-video R@1· uses extra data· 2022-09-04
    37.2
    best: 64 (GRAM)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video RetrievalonMSR-VTT
    text-to-video R@10· uses extra data· 2022-09-04
    75.8
    best: 89.6 (VAST)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video RetrievalonMSR-VTT
    text-to-video R@5· uses extra data· 2022-09-04
    64.8
    best: 84.3 (VAST)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video RetrievalonLSMDC
    text-to-video R@1· 2022-09-04
    24
    best: 46.4 (InternVideo2-6B)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video RetrievalonLSMDC
    text-to-video R@10· 2022-09-04
    54.1
    best: 92.8 (HunYuan_tvr (huge))
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video RetrievalonLSMDC
    text-to-video R@5· 2022-09-04
    43.5
    best: 80.1 (HunYuan_tvr (huge))
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540

Reasoning3 results

  • Video Question AnsweringonLSMDC-MC
    Accuracy· 2022-09-04
    84.4
    SOTA
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video Question AnsweringonMSRVTT-MC
    Accuracy· 2022-09-04
    97.6
    SOTA
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540
  • Video Question AnsweringonMSRVTT-QA
    Accuracy· 2022-09-04
    44.5
    best: 72.4 (Flash-VStream)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540

Natural Language Processing1 result

  • Visual Question Answering (VQA)onMSVD-QA
    Accuracy· uses extra data· 2022-09-04
    0.547
    best: 0.61 (VLAB)
    An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingarXiv:2209.01540