TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/SURE: SUrvey REcipes for building reliable and robust deep...

SURE: SUrvey REcipes for building reliable and robust deep networks

Yuting Li, Yingyi Chen, Xuanlong Yu, Dexiong Chen, Xi Shen

2024-03-01CVPR 2024 1Image ClassificationLong-tail LearningLearning with noisy labels
PaperPDFCode(official)

Abstract

In this paper, we revisit techniques for uncertainty estimation within deep neural networks and consolidate a suite of techniques to enhance their reliability. Our investigation reveals that an integrated application of diverse techniques--spanning model regularization, classifier and optimization--substantially improves the accuracy of uncertainty predictions in image classification tasks. The synergistic effect of these techniques culminates in our novel SURE approach. We rigorously evaluate SURE against the benchmark of failure prediction, a critical testbed for uncertainty estimation efficacy. Our results showcase a consistently better performance than models that individually deploy each technique, across various datasets and model architectures. When applied to real-world challenges, such as data corruption, label noise, and long-tailed class distribution, SURE exhibits remarkable robustness, delivering results that are superior or on par with current state-of-the-art specialized methods. Particularly on Animal-10N and Food-101N for learning with noisy labels, SURE achieves state-of-the-art performance without any task-specific adjustments. This work not only sets a new benchmark for robust uncertainty estimation but also paves the way for its application in diverse, real-world scenarios where reliability is paramount. Our code is available at \url{https://yutingli0606.github.io/SURE/}.

Results

TaskDatasetMetricValueModel
Image ClassificationFood-101NAccuracy88SURE(ResNet-50)
Image ClassificationCIFAR-10-LT (ρ=10)Error Rate5.04SURE(ResNet-32)
Image ClassificationCIFAR-100-LT (ρ=50)Error Rate36.87SURE(ResNet-32)
Image ClassificationCIFAR-100-LT (ρ=10)Error Rate26.76SURE(ResNet-32)
Image ClassificationCIFAR-10-LT (ρ=50)Error Rate9.78SURE(ResNet-32)
Image ClassificationCIFAR-100-LT (ρ=100)Error Rate43.66SURE(ResNet-32)
Image ClassificationCIFAR-10-LT (ρ=100)Error Rate13.07SURE(ResNet-32)
Image ClassificationANIMALAccuracy89SURE
Document Text ClassificationANIMALAccuracy89SURE
Few-Shot Image ClassificationCIFAR-10-LT (ρ=10)Error Rate5.04SURE(ResNet-32)
Few-Shot Image ClassificationCIFAR-100-LT (ρ=50)Error Rate36.87SURE(ResNet-32)
Few-Shot Image ClassificationCIFAR-100-LT (ρ=10)Error Rate26.76SURE(ResNet-32)
Few-Shot Image ClassificationCIFAR-10-LT (ρ=50)Error Rate9.78SURE(ResNet-32)
Few-Shot Image ClassificationCIFAR-100-LT (ρ=100)Error Rate43.66SURE(ResNet-32)
Few-Shot Image ClassificationCIFAR-10-LT (ρ=100)Error Rate13.07SURE(ResNet-32)
Generalized Few-Shot ClassificationCIFAR-10-LT (ρ=10)Error Rate5.04SURE(ResNet-32)
Generalized Few-Shot ClassificationCIFAR-100-LT (ρ=50)Error Rate36.87SURE(ResNet-32)
Generalized Few-Shot ClassificationCIFAR-100-LT (ρ=10)Error Rate26.76SURE(ResNet-32)
Generalized Few-Shot ClassificationCIFAR-10-LT (ρ=50)Error Rate9.78SURE(ResNet-32)
Generalized Few-Shot ClassificationCIFAR-100-LT (ρ=100)Error Rate43.66SURE(ResNet-32)
Generalized Few-Shot ClassificationCIFAR-10-LT (ρ=100)Error Rate13.07SURE(ResNet-32)
Long-tail LearningCIFAR-10-LT (ρ=10)Error Rate5.04SURE(ResNet-32)
Long-tail LearningCIFAR-100-LT (ρ=50)Error Rate36.87SURE(ResNet-32)
Long-tail LearningCIFAR-100-LT (ρ=10)Error Rate26.76SURE(ResNet-32)
Long-tail LearningCIFAR-10-LT (ρ=50)Error Rate9.78SURE(ResNet-32)
Long-tail LearningCIFAR-100-LT (ρ=100)Error Rate43.66SURE(ResNet-32)
Long-tail LearningCIFAR-10-LT (ρ=100)Error Rate13.07SURE(ResNet-32)
Generalized Few-Shot LearningCIFAR-10-LT (ρ=10)Error Rate5.04SURE(ResNet-32)
Generalized Few-Shot LearningCIFAR-100-LT (ρ=50)Error Rate36.87SURE(ResNet-32)
Generalized Few-Shot LearningCIFAR-100-LT (ρ=10)Error Rate26.76SURE(ResNet-32)
Generalized Few-Shot LearningCIFAR-10-LT (ρ=50)Error Rate9.78SURE(ResNet-32)
Generalized Few-Shot LearningCIFAR-100-LT (ρ=100)Error Rate43.66SURE(ResNet-32)
Generalized Few-Shot LearningCIFAR-10-LT (ρ=100)Error Rate13.07SURE(ResNet-32)

Related Papers

Automatic Classification and Segmentation of Tunnel Cracks Based on Deep Learning and Visual Explanations2025-07-18Adversarial attacks to image classification systems using evolutionary algorithms2025-07-17Efficient Adaptation of Pre-trained Vision Transformer underpinned by Approximately Orthogonal Fine-Tuning Strategy2025-07-17Federated Learning for Commercial Image Sources2025-07-17MUPAX: Multidimensional Problem Agnostic eXplainable AI2025-07-17CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels2025-07-16Hashed Watermark as a Filter: Defeating Forging and Overwriting Attacks in Weight-based Neural Network Watermarking2025-07-15Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networks2025-07-14