Aurora (ours, r=128)

Reported on 36 benchmarks across 3 tasks

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Miscellaneous24 results

Image Retrieval with Multi-Modal QueryonFlickr30k
Image-to-text R@1· uses extra data
97.2
best: 98.8 (X2-VLM (large))
Image Retrieval with Multi-Modal QueryonFlickr30k
Image-to-text R@10· uses extra data
100
Image Retrieval with Multi-Modal QueryonFlickr30k
Image-to-text R@5· uses extra data
100
Image Retrieval with Multi-Modal QueryonFlickr30k
Text-to-image R@1· uses extra data
86.8
best: 93.3 (ERNIE-ViL 2.0)
Image Retrieval with Multi-Modal QueryonFlickr30k
Text-to-image R@10· uses extra data
98.9
best: 99.8 (ERNIE-ViL 2.0)
Image Retrieval with Multi-Modal QueryonFlickr30k
Text-to-image R@5· uses extra data
97.6
best: 99.5 (M2-Encoder)
Image Retrieval with Multi-Modal QueryonCOCO 2014
Image-to-text R@1· uses extra data
80.7
best: 84.8 (BEiT-3)
Image Retrieval with Multi-Modal QueryonCOCO 2014
Image-to-text R@10· uses extra data
97.8
best: 98.5 (X2-VLM (large))
Image Retrieval with Multi-Modal QueryonCOCO 2014
Image-to-text R@5· uses extra data
95.3
best: 96.5 (X2-VLM (large))
Image Retrieval with Multi-Modal QueryonCOCO 2014
Text-to-image R@1· uses extra data
62.8
best: 68 (VAST)
Image Retrieval with Multi-Modal QueryonCOCO 2014
Text-to-image R@10· uses extra data
91
best: 92.8 (VAST)
Image Retrieval with Multi-Modal QueryonCOCO 2014
Text-to-image R@5· uses extra data
84.8
best: 92.8 (BEiT-3)
Cross-Modal Information RetrievalonFlickr30k
Image-to-text R@1· uses extra data
97.2
best: 98.8 (X2-VLM (large))
Cross-Modal Information RetrievalonFlickr30k
Image-to-text R@10· uses extra data
100
Cross-Modal Information RetrievalonFlickr30k
Image-to-text R@5· uses extra data
100
Cross-Modal Information RetrievalonFlickr30k
Text-to-image R@1· uses extra data
86.8
best: 93.3 (ERNIE-ViL 2.0)
Cross-Modal Information RetrievalonFlickr30k
Text-to-image R@10· uses extra data
98.9
best: 99.8 (ERNIE-ViL 2.0)
Cross-Modal Information RetrievalonFlickr30k
Text-to-image R@5· uses extra data
97.6
best: 99.4 (ERNIE-ViL 2.0)
Cross-Modal Information RetrievalonCOCO 2014
Image-to-text R@1· uses extra data
80.7
best: 84.8 (BEiT-3)
Cross-Modal Information RetrievalonCOCO 2014
Image-to-text R@10· uses extra data
97.8
best: 98.5 (X2-VLM (large))
Cross-Modal Information RetrievalonCOCO 2014
Image-to-text R@5· uses extra data
95.3
best: 96.5 (X2-VLM (large))
Cross-Modal Information RetrievalonCOCO 2014
Text-to-image R@1· uses extra data
62.8
best: 68 (VAST)
Cross-Modal Information RetrievalonCOCO 2014
Text-to-image R@10· uses extra data
91
best: 92.8 (VAST)
Cross-Modal Information RetrievalonCOCO 2014
Text-to-image R@5· uses extra data
84.8
best: 92.8 (BEiT-3)

Natural Language Processing12 results

Cross-Modal RetrievalonFlickr30k
Image-to-text R@1· uses extra data
97.2
best: 98.8 (X2-VLM (large))
Cross-Modal RetrievalonFlickr30k
Image-to-text R@10· uses extra data
100
Cross-Modal RetrievalonFlickr30k
Image-to-text R@5· uses extra data
100
Cross-Modal RetrievalonFlickr30k
Text-to-image R@1· uses extra data
86.8
best: 93.3 (ERNIE-ViL 2.0)
Cross-Modal RetrievalonFlickr30k
Text-to-image R@10· uses extra data
98.9
best: 99.8 (ERNIE-ViL 2.0)
Cross-Modal RetrievalonFlickr30k
Text-to-image R@5· uses extra data
97.6
best: 99.4 (ERNIE-ViL 2.0)
Cross-Modal RetrievalonCOCO 2014
Image-to-text R@1· uses extra data
80.7
best: 84.8 (BEiT-3)
Cross-Modal RetrievalonCOCO 2014
Image-to-text R@10· uses extra data
97.8
best: 98.5 (X2-VLM (large))
Cross-Modal RetrievalonCOCO 2014
Image-to-text R@5· uses extra data
95.3
best: 96.5 (X2-VLM (large))
Cross-Modal RetrievalonCOCO 2014
Text-to-image R@1· uses extra data
62.8
best: 68 (VAST)
Cross-Modal RetrievalonCOCO 2014
Text-to-image R@10· uses extra data
91
best: 92.8 (VAST)
Cross-Modal RetrievalonCOCO 2014
Text-to-image R@5· uses extra data
84.8
best: 92.8 (BEiT-3)