Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/OmniVec

OmniVec

Reported on 32 benchmarks across 14 tasks · 1 paper · 22 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Vision16 results

VideoonMoments in Time
Top 1 Accuracy· uses extra data· 2023-11-07
49.8
best: 53.1 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
VideoonMIT
Top 1 Accuracy· 2023-11-07
49.8
best: 51.2 (InternVideo2-6B)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Image ClassificationoniNaturalist 2018
Top-1 Accuracy· uses extra data· 2023-11-07
93.8
best: 94.6 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Image ClassificationonOxford-IIIT Pet Dataset
Accuracy· uses extra data· 2023-11-07
99.2
best: 99.6 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Shape Representation Of 3D Point CloudsonScanObjectNN
Overall Accuracy· uses extra data· 2023-11-07
96.1
best: 97.2 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Shape Representation Of 3D Point CloudsonModelNet40-C
Error Rate· uses extra data· 2023-11-07
0.156
best: 0.142 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Fine-Grained Image ClassificationonOxford-IIIT Pet Dataset
Accuracy· uses extra data· 2023-11-07
99.2
best: 99.6 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
3D Point Cloud ClassificationonScanObjectNN
Overall Accuracy· uses extra data· 2023-11-07
96.1
best: 97.2 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
3D Point Cloud ClassificationonModelNet40-C
Error Rate· uses extra data· 2023-11-07
0.156
best: 0.142 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
3D Point Cloud ReconstructiononScanObjectNN
Overall Accuracy· uses extra data· 2023-11-07
96.1
best: 97.2 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
3D Point Cloud ReconstructiononModelNet40-C
Error Rate· uses extra data· 2023-11-07
0.156
best: 0.142 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
VideoonMSR-VTT-1kA
text-to-video R@10· uses extra data· 2023-11-07
89.4
best: 90.8 (HunYuan_tvr (huge))
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
VideoonYouCook2
text-to-video R@10· uses extra data· 2023-11-07
70.8
best: 80.8 (VAST)
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
VideoonKinetics-400
Acc@1· 2023-11-07
91.1
best: 93.6 (OmniVec2)
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Video RetrievalonMSR-VTT-1kA
text-to-video R@10· uses extra data· 2023-11-07
89.4
best: 90.8 (HunYuan_tvr (huge))
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Video RetrievalonYouCook2
text-to-video R@10· uses extra data· 2023-11-07
70.8
best: 80.8 (VAST)
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709

Audio5 results

Audio ClassificationonESC-50
Accuracy (5-fold)· uses extra data· 2023-11-07
98.4
best: 99.1 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Audio ClassificationonESC-50
Top-1 Accuracy· uses extra data· 2023-11-07
98.4
best: 99.1 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Audio ClassificationonAudioSet
Test mAP· uses extra data· 2023-11-07
0.548
best: 0.558 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
10-shot image generationonNYU Depth v2
Mean IoU· uses extra data· 2023-11-07
60.8
best: 63.6 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
10-shot image generationonS3DIS Area5
mIoU· uses extra data· 2023-11-07
75.9
best: 76 (Sonata + PTv3)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709

Knowledge Base4 results

Text SummarizationonDialogSum
BertScore· uses extra data· 2023-11-07
71.91
best: 72.8 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Text SummarizationonDialogSum
Rouge1· uses extra data· 2023-11-07
46.91
best: 47.8 (InstructDS)
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Text SummarizationonDialogSum
Rouge2· uses extra data· 2023-11-07
21.22
best: 22.2 (InstructDS)
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Text SummarizationonDialogSum
RougeL· uses extra data· 2023-11-07
40.19
best: 41.4 (OmniVec2)
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709

Methodology3 results

ClassificationonESC-50
Accuracy (5-fold)· uses extra data· 2023-11-07
98.4
best: 99.1 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
ClassificationonESC-50
Top-1 Accuracy· uses extra data· 2023-11-07
98.4
best: 99.1 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
ClassificationonAudioSet
Test mAP· uses extra data· 2023-11-07
0.548
best: 0.558 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709

Medical2 results

Semantic SegmentationonNYU Depth v2
Mean IoU· uses extra data· 2023-11-07
60.8
best: 63.6 (OmniVec2)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709
Semantic SegmentationonS3DIS Area5
mIoU· uses extra data· 2023-11-07
75.9
best: 76 (Sonata + PTv3)
SOTA
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709

Robots1 result

Activity RecognitiononUCF101
3-fold Accuracy· uses extra data· 2023-11-07
99.6
best: 99.7 (FTP-UniFormerV2-L/14)
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709

Time Series1 result

Action RecognitiononUCF101
3-fold Accuracy· uses extra data· 2023-11-07
99.6
best: 99.7 (FTP-UniFormerV2-L/14)
OmniVec: Learning robust representations with cross modal sharing arXiv:2311.05709