TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/VideoChat2_HD_mistral

VideoChat2_HD_mistral

Reported on 31 benchmarks across 6 tasks · 1 paper · 23 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Reasoning29 results

  • Video Question AnsweringonNExT-QA
    Accuracy· 2023-11-28
    79.5
    best: 85.5 (LinVT-Qwen2-VL (7B))
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video Question AnsweringonIntentQA
    Accuarcy· 2023-11-28
    83.4
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video Question AnsweringonIntentQA
    CH· 2023-11-28
    90
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video Question AnsweringonIntentQA
    CW· 2023-11-28
    84
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video Question AnsweringonIntentQA
    TP&TN· 2023-11-28
    77.3
    best: 79.1 (Human)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video Question AnsweringonEgoSchema (subset)
    Accuracy· 2023-11-28
    65.6
    best: 68.6 (Tarsier (34B))
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    Consistency· 2023-11-28
    2.84
    best: 3.81 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    Contextual Understanding· 2023-11-28
    3.72
    best: 4.21 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    Correctness of Information· 2023-11-28
    3.4
    best: 3.85 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    mean· 2023-11-28
    3.1
    best: 3.73 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    gpt-score· 2023-11-28
    3.64
    best: 4.21 (PPLLaVA-7B)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    Consistency· 2023-11-28
    2.84
    best: 3.81 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    Contextual Understanding· 2023-11-28
    3.72
    best: 4.21 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    Correctness of Information· 2023-11-28
    3.4
    best: 3.85 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    mean· 2023-11-28
    3.1
    best: 3.73 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    gpt-score· 2023-11-28
    3.64
    best: 4.21 (PPLLaVA-7B)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video Question AnsweringonEgoSchema (fullset)
    Accuracy· 2023-11-28
    55.8
    best: 71.14 (BIMBA-LLaVA-Qwen2-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    Detail Orientation· 2023-11-28
    2.91
    best: 3.56 (PPLLaVA-7B-dpo)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    Temporal Understanding· 2023-11-28
    2.65
    best: 3.23 (VLM-RLAIF)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    gpt-score· 2023-11-28
    3.4
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    gpt-score· 2023-11-28
    2.86
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    gpt-score· 2023-11-28
    2.65
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Generative Visual Question AnsweringonVideoInstruct
    gpt-score· 2023-11-28
    2.62
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    Detail Orientation· 2023-11-28
    2.91
    best: 3.56 (PPLLaVA-7B-dpo)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    Temporal Understanding· 2023-11-28
    2.65
    best: 3.23 (VLM-RLAIF)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    gpt-score· 2023-11-28
    3.4
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    gpt-score· 2023-11-28
    2.86
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    gpt-score· 2023-11-28
    2.65
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Video-based Generative Performance BenchmarkingonVideoInstruct
    gpt-score· 2023-11-28
    2.62
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005

Natural Language Processing13 results

  • Question AnsweringonEgoSchema (subset)
    Accuracy· 2023-11-28
    65.6
    best: 68.6 (Tarsier (34B))
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    Consistency· 2023-11-28
    2.84
    best: 3.81 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    Contextual Understanding· 2023-11-28
    3.72
    best: 4.21 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    Correctness of Information· 2023-11-28
    3.4
    best: 3.85 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    mean· 2023-11-28
    3.1
    best: 3.73 (PPLLaVA-7B-dpo)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    gpt-score· 2023-11-28
    3.64
    best: 4.21 (PPLLaVA-7B)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Question AnsweringonEgoSchema (fullset)
    Accuracy· 2023-11-28
    55.8
    best: 71.14 (BIMBA-LLaVA-Qwen2-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    Detail Orientation· 2023-11-28
    2.91
    best: 3.56 (PPLLaVA-7B-dpo)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    Temporal Understanding· 2023-11-28
    2.65
    best: 3.23 (VLM-RLAIF)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    gpt-score· 2023-11-28
    3.4
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    gpt-score· 2023-11-28
    2.86
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    gpt-score· 2023-11-28
    2.65
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005
  • Visual Question Answering (VQA)onVideoInstruct
    gpt-score· 2023-11-28
    2.62
    best: 4.21 (PPLLaVA-7B)
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005

Other1 result

  • Video-based Generative Performance Benchmarking (Correctness of Information)onVideoInstruct
    gpt-score· 2023-11-28
    3.4
    best: 3.85 (PPLLaVA-7B)
    SOTA
    MVBench: A Comprehensive Multi-modal Video Understanding BenchmarkarXiv:2311.17005