Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/COSMOS ViT-B/16

COSMOS ViT-B/16

Reported on 31 benchmarks across 5 tasks · 1 paper · 1 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Miscellaneous12 results

Image Retrieval with Multi-Modal QueryonFlickr30k
Image-to-text R@1· 2024-12-02
92.9
best: 98.8 (X2-VLM (large))
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonFlickr30k
Image-to-text R@10· 2024-12-02
99.9
best: 100 (X2-VLM (large))
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonFlickr30k
Image-to-text R@5· 2024-12-02
99.4
best: 100 (X2-VLM (large))
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonFlickr30k
Text-to-image R@1· 2024-12-02
80.3
best: 93.3 (ERNIE-ViL 2.0)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonFlickr30k
Text-to-image R@10· 2024-12-02
97.6
best: 99.8 (ERNIE-ViL 2.0)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonFlickr30k
Text-to-image R@5· 2024-12-02
95.3
best: 99.5 (M2-Encoder)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonCOCO 2014
Image-to-text R@1· 2024-12-02
68
best: 84.8 (BEiT-3)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonCOCO 2014
Image-to-text R@10· 2024-12-02
92.5
best: 98.5 (X2-VLM (large))
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonCOCO 2014
Image-to-text R@5· 2024-12-02
87.8
best: 96.5 (X2-VLM (large))
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonCOCO 2014
Text-to-image R@1· 2024-12-02
52.5
best: 68 (VAST)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonCOCO 2014
Text-to-image R@10· 2024-12-02
84.9
best: 92.8 (VAST)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Image Retrieval with Multi-Modal QueryonCOCO 2014
Text-to-image R@5· 2024-12-02
77.2
best: 92.8 (BEiT-3)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814

Computer Vision7 results

Zero Shot SegmentationonADE20K training-free zero-shot segmentation
mIoU· 2024-12-02
17.7
SOTA
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Unsupervised Semantic SegmentationonCOCO-Stuff-171
mIoU· 2024-12-02
23.2
best: 34 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Unsupervised Semantic SegmentationonCOCO-Object
mIoU· 2024-12-02
31.3
best: 49.4 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Unsupervised Semantic SegmentationonADE20K
Mean IoU (val)· 2024-12-02
17.7
best: 30.7 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Unsupervised Semantic SegmentationonCityscapes val
mIoU· 2024-12-02
34.7
best: 51.1 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Unsupervised Semantic SegmentationonPASCAL Context-59
mIoU· 2024-12-02
33.7
best: 50.8 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Unsupervised Semantic SegmentationonPascalVOC-20
mIoU· 2024-12-02
77.7
best: 91.8 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814

Medical6 results

Semantic SegmentationonCOCO-Stuff-171
mIoU· 2024-12-02
23.2
best: 34 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Semantic SegmentationonCOCO-Object
mIoU· 2024-12-02
31.3
best: 49.4 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Semantic SegmentationonADE20K
Mean IoU (val)· 2024-12-02
17.7
best: 30.7 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Semantic SegmentationonCityscapes val
mIoU· 2024-12-02
34.7
best: 90.3 (EfficientPS (Cityscapes-fine))
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Semantic SegmentationonPASCAL Context-59
mIoU· 2024-12-02
33.7
best: 50.8 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
Semantic SegmentationonPascalVOC-20
mIoU· 2024-12-02
77.7
best: 91.8 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814

Audio6 results

10-shot image generationonCOCO-Stuff-171
mIoU· 2024-12-02
23.2
best: 34 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
10-shot image generationonCOCO-Object
mIoU· 2024-12-02
31.3
best: 49.4 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
10-shot image generationonADE20K
Mean IoU (val)· 2024-12-02
17.7
best: 30.7 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
10-shot image generationonCityscapes val
mIoU· 2024-12-02
34.7
best: 90.3 (EfficientPS (Cityscapes-fine))
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
10-shot image generationonPASCAL Context-59
mIoU· 2024-12-02
33.7
best: 50.8 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814
10-shot image generationonPascalVOC-20
mIoU· 2024-12-02
77.7
best: 91.8 (CorrCLIP)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training arXiv:2412.01814