Gemini-Pro 4-shot

Reported on 2 benchmarks across 2 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing1 result

Visual Question Answering (VQA)onIllusionVQA
Accuracy· 2024-03-23
52.87
best: 62.99 (GPT4-Vision 4-shot)
IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models arXiv:2403.15952

Computer Vision1 result

Object LocalizationonIllusionVQA
Accuracy· 2024-03-23
41.8
best: 49.7 (GPT4-Vision 4-shot+CoT)
IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models arXiv:2403.15952