Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/Tarsier (34B)

Tarsier (34B)

Reported on 23 benchmarks across 2 tasks · 1 paper · 8 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Reasoning12 results

Video Question AnsweringonMVBench
Avg.· 2024-06-30
67.6
best: 69.3 (LinVT-Qwen2-VL (7B))
SOTA
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonTGIF-QA
Accuracy· 2024-06-30
82.5
SOTA
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonEgoSchema (subset)
Accuracy· 2024-06-30
68.6
SOTA
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonActivityNet-QA
Accuracy· 2024-06-30
61.6
SOTA
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonNExT-QA
Accuracy· 2024-06-30
79.2
best: 85.5 (LinVT-Qwen2-VL (7B))
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonMSVD-QA
Accuracy· 2024-06-30
80.3
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonMSVD-QA
Confidence Score· 2024-06-30
4.2
best: 2.5 (Video LLaMA-7B)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonTGIF-QA
Confidence Score· 2024-06-30
4.4
best: 2.3 (Video Chat-7B)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonMSRVTT-QA
Accuracy· 2024-06-30
66.4
best: 72.4 (Flash-VStream)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonMSRVTT-QA
Confidence Score· 2024-06-30
3.7
best: 1.8 (Video LLaMA-7B)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonEgoSchema (fullset)
Accuracy· 2024-06-30
61.7
best: 71.14 (BIMBA-LLaVA-Qwen2-7B)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Video Question AnsweringonActivityNet-QA
Confidence Score· 2024-06-30
3.7
best: 1.1 (Video LLaMA)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634

Natural Language Processing11 results

Question AnsweringonNExT-QA
Accuracy· 2024-06-30
79.2
best: 79.6 (VideoMultiAgent (GPT-4o))
SOTA
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonTGIF-QA
Accuracy· 2024-06-30
82.5
SOTA
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonEgoSchema (subset)
Accuracy· 2024-06-30
68.6
SOTA
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonActivityNet-QA
Accuracy· 2024-06-30
61.6
SOTA
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonMSVD-QA
Accuracy· 2024-06-30
80.3
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonMSVD-QA
Confidence Score· 2024-06-30
4.2
best: 2.5 (Video LLaMA-7B)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonTGIF-QA
Confidence Score· 2024-06-30
4.4
best: 2.3 (Video Chat-7B)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonMSRVTT-QA
Accuracy· 2024-06-30
66.4
best: 72.4 (Flash-VStream)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonMSRVTT-QA
Confidence Score· 2024-06-30
3.7
best: 1.8 (Video LLaMA-7B)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonEgoSchema (fullset)
Accuracy· 2024-06-30
61.7
best: 71.14 (BIMBA-LLaVA-Qwen2-7B)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634
Question AnsweringonActivityNet-QA
Confidence Score· 2024-06-30
3.7
best: 1.1 (Video LLaMA)
Tarsier: Recipes for Training and Evaluating Large Video Description Models arXiv:2407.00634