Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/SimVLM

SimVLM

Reported on 14 benchmarks across 4 tasks · 1 paper · 13 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing12 results

Visual Question Answering (VQA)onVQA v2 test-dev
Accuracy· 2021-08-24
80.03
best: 84.3 (PaLI)
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Visual Question Answering (VQA)onVQA v2 test-std
overall· 2021-08-24
80.34
best: 84.03 (BEiT-3)
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Natural Language InferenceonSNLI-VE val
Accuracy· 2021-08-24
86.21
best: 91 (OFA)
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Natural Language InferenceonSNLI-VE test
Accuracy· 2021-08-24
86.32
best: 91.2 (OFA)
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Image Captioningonnocaps-val-out-domain
CIDEr· 2021-08-24
115.2
best: 124.8 (BLIP-2 ViT-G FlanT5 XL (zero-shot))
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Image Captioningonnocaps-val-near-domain
CIDEr· 2021-08-24
110.9
best: 120.2 (BLIP-2 ViT-G FlanT5 XL (zero-shot))
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Image CaptioningonCOCO Captions
CIDER· 2021-08-24
143.3
best: 155.1 (mPLUG)
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Image CaptioningonCOCO Captions
METEOR· 2021-08-24
33.4
best: 33.9 (CoCa)
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Image CaptioningonCOCO Captions
SPICE· 2021-08-24
25.4
best: 27 (VAST)
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Image Captioningonnocaps-val-overall
CIDEr· 2021-08-24
112.2
best: 121.6 (BLIP-2 ViT-G FlanT5 XL (zero-shot))
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Image Captioningonnocaps-val-in-domain
CIDEr· 2021-08-24
113.7
best: 123.7 (BLIP-2 ViT-G FlanT5 XL (zero-shot))
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Image CaptioningonCOCO Captions
BLEU-4· 2021-08-24
40.6
best: 46.5 (mPLUG)
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904

Reasoning2 results

Visual ReasoningonNLVR2 Dev
Accuracy· 2021-08-24
84.53
best: 91.51 (BEiT-3)
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904
Visual ReasoningonNLVR2 Test
Accuracy· 2021-08-24
85.15
best: 92.58 (BEiT-3)
SOTA
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904