Gemini Ultra

Reported on 3 benchmarks across 2 tasks · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing2 results

Visual Question Answering (VQA)onChartQA
1:1 Accuracy· 2023-12-19
80.8
best: 81.3 (ChartPaLI-5B + PaLM 2-S)
SOTA
Gemini: A Family of Highly Capable Multimodal Models arXiv:2312.11805
Visual Question Answering (VQA)onAI2D
EM· 2023-12-19
79.5
best: 82.5 (SMoLA-PaLI-X Specialist Model)
Gemini: A Family of Highly Capable Multimodal Models arXiv:2312.11805

Computer Code1 result

Chart Question AnsweringonChartQA
1:1 Accuracy· 2023-12-19
80.8
best: 81.3 (ChartPaLI-5B + PaLM 2-S)
SOTA
Gemini: A Family of Highly Capable Multimodal Models arXiv:2312.11805