Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/XFM (base)

XFM (base)

Reported on 24 benchmarks across 6 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Miscellaneous12 results

Image Retrieval with Multi-Modal QueryonCOCO 2014
Image-to-text R@1· uses extra data· 2023-01-12
84.2
best: 84.8 (BEiT-3)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Image Retrieval with Multi-Modal QueryonCOCO 2014
Image-to-text R@10· uses extra data· 2023-01-12
98.4
best: 98.5 (X2-VLM (large))
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Image Retrieval with Multi-Modal QueryonCOCO 2014
Image-to-text R@5· uses extra data· 2023-01-12
96.4
best: 96.5 (X2-VLM (large))
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Image Retrieval with Multi-Modal QueryonCOCO 2014
Text-to-image R@1· uses extra data· 2023-01-12
67
best: 68 (VAST)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Image Retrieval with Multi-Modal QueryonCOCO 2014
Text-to-image R@10· uses extra data· 2023-01-12
92.4
best: 92.8 (VAST)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Image Retrieval with Multi-Modal QueryonCOCO 2014
Text-to-image R@5· uses extra data· 2023-01-12
87.2
best: 92.8 (BEiT-3)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal Information RetrievalonCOCO 2014
Image-to-text R@1· uses extra data· 2023-01-12
84.2
best: 84.8 (BEiT-3)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal Information RetrievalonCOCO 2014
Image-to-text R@10· uses extra data· 2023-01-12
98.4
best: 98.5 (X2-VLM (large))
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal Information RetrievalonCOCO 2014
Image-to-text R@5· uses extra data· 2023-01-12
96.4
best: 96.5 (X2-VLM (large))
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal Information RetrievalonCOCO 2014
Text-to-image R@1· uses extra data· 2023-01-12
67
best: 68 (VAST)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal Information RetrievalonCOCO 2014
Text-to-image R@10· uses extra data· 2023-01-12
92.4
best: 92.8 (VAST)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal Information RetrievalonCOCO 2014
Text-to-image R@5· uses extra data· 2023-01-12
87.2
best: 92.8 (BEiT-3)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065

Natural Language Processing7 results

Visual Question Answering (VQA)onVQA v2 test-dev
Accuracy· 2023-01-12
80.4
best: 84.3 (PaLI)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal RetrievalonCOCO 2014
Image-to-text R@1· uses extra data· 2023-01-12
84.2
best: 84.8 (BEiT-3)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal RetrievalonCOCO 2014
Image-to-text R@10· uses extra data· 2023-01-12
98.4
best: 98.5 (X2-VLM (large))
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal RetrievalonCOCO 2014
Image-to-text R@5· uses extra data· 2023-01-12
96.4
best: 96.5 (X2-VLM (large))
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal RetrievalonCOCO 2014
Text-to-image R@1· uses extra data· 2023-01-12
67
best: 68 (VAST)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal RetrievalonCOCO 2014
Text-to-image R@10· uses extra data· 2023-01-12
92.4
best: 92.8 (VAST)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Cross-Modal RetrievalonCOCO 2014
Text-to-image R@5· uses extra data· 2023-01-12
87.2
best: 92.8 (BEiT-3)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065

Computer Vision3 results

Visual GroundingonRefCOCO+ test B
Accuracy (%)· 2023-01-12
79.8
best: 92 (Florence-2-large-ft)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Visual GroundingonRefCOCO+ val
Accuracy (%)· 2023-01-12
86.1
best: 93.4 (Florence-2-large-ft)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Visual GroundingonRefCOCO+ testA
Accuracy (%)· 2023-01-12
90.4
best: 95.3 (Florence-2-large-ft)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065

Reasoning2 results

Visual ReasoningonNLVR2 Dev
Accuracy· 2023-01-12
87.6
best: 91.51 (BEiT-3)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065
Visual ReasoningonNLVR2 Test
Accuracy· 2023-01-12
88.4
best: 92.58 (BEiT-3)
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks arXiv:2301.05065