TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/LangRepo (12B)

LangRepo (12B)

Reported on 10 benchmarks across 2 tasks · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing5 results

  • Question AnsweringonEgoSchema (subset)
    Accuracy· 2024-03-21
    66.2
    best: 68.6 (Tarsier (34B))
    SOTA
    Language Repository for Long Video UnderstandingarXiv:2403.14622
  • Question AnsweringonNExT-QA
    Accuracy· 2024-03-21
    60.9
    best: 79.6 (VideoMultiAgent (GPT-4o))
    Language Repository for Long Video UnderstandingarXiv:2403.14622
  • Question AnsweringonNExT-GQA
    Acc@GQA· 2024-03-21
    17.1
    best: 28.9 (DeVi (Gemini 2.0))
    Language Repository for Long Video UnderstandingarXiv:2403.14622
  • Question AnsweringonIntentQA
    Accuracy· 2024-03-21
    59.1
    best: 71.5 (ENTER)
    Language Repository for Long Video UnderstandingarXiv:2403.14622
  • Question AnsweringonEgoSchema (fullset)
    Accuracy· 2024-03-21
    41.2
    best: 71.14 (BIMBA-LLaVA-Qwen2-7B)
    Language Repository for Long Video UnderstandingarXiv:2403.14622

Reasoning5 results

  • Video Question AnsweringonEgoSchema (subset)
    Accuracy· 2024-03-21
    66.2
    best: 68.6 (Tarsier (34B))
    SOTA
    Language Repository for Long Video UnderstandingarXiv:2403.14622
  • Video Question AnsweringonNExT-QA
    Accuracy· 2024-03-21
    60.9
    best: 85.5 (LinVT-Qwen2-VL (7B))
    Language Repository for Long Video UnderstandingarXiv:2403.14622
  • Video Question AnsweringonNExT-GQA
    Acc@GQA· 2024-03-21
    17.1
    best: 28.9 (DeVi (Gemini 2.0))
    Language Repository for Long Video UnderstandingarXiv:2403.14622
  • Video Question AnsweringonIntentQA
    Accuracy· 2024-03-21
    59.1
    best: 71.5 (ENTER)
    Language Repository for Long Video UnderstandingarXiv:2403.14622
  • Video Question AnsweringonEgoSchema (fullset)
    Accuracy· 2024-03-21
    41.2
    best: 71.14 (BIMBA-LLaVA-Qwen2-7B)
    Language Repository for Long Video UnderstandingarXiv:2403.14622