TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/Single Layer Predictive Normalized Maximum Likelihood for ...

Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection

Koby Bibas, Meir Feder, Tal Hassner

2021-10-18NeurIPS 2021 12Out of Distribution (OOD) DetectionOut-of-Distribution Detection
PaperPDFCode(official)

Abstract

Detecting out-of-distribution (OOD) samples is vital for developing machine learning based models for critical safety systems. Common approaches for OOD detection assume access to some OOD samples during training which may not be available in a real-life scenario. Instead, we utilize the {\em predictive normalized maximum likelihood} (pNML) learner, in which no assumptions are made on the tested input. We derive an explicit expression of the pNML and its generalization error, denoted as the {\em regret}, for a single layer neural network (NN). We show that this learner generalizes well when (i) the test vector resides in a subspace spanned by the eigenvectors associated with the large eigenvalues of the empirical correlation matrix of the training data, or (ii) the test sample is far from the decision boundary. Furthermore, we describe how to efficiently apply the derived pNML regret to any pretrained deep NN, by employing the explicit pNML for the last layer, followed by the softmax function. Applying the derived regret to deep NN requires neither additional tunable parameters nor extra data. We extensively evaluate our approach on 74 OOD detection benchmarks using DenseNet-100, ResNet-34, and WideResNet-40 models trained with CIFAR-100, CIFAR-10, SVHN, and ImageNet-30 showing a significant improvement of up to 15.6\% over recent leading methods.

Results

TaskDatasetMetricValueModel
Out-of-Distribution DetectionCIFAR-10 vs GaussianAUROC100DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-10 vs GaussianAUROC100ResNet-34
Out-of-Distribution DetectionCIFAR-10 vs UniformAUROC100DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-10 vs UniformAUROC100ResNet-34
Out-of-Distribution DetectionCIFAR-10 vs LSUN (C)AUROC99.9DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-10 vs LSUN (C)AUROC99.5ResNet-34
Out-of-Distribution DetectionSVHN vs iSUNAUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs iSUNAUROC100ResNet-34
Out-of-Distribution DetectionCIFAR-100 vs ImageNet (C)AUROC99DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-100 vs ImageNet (C)AUROC98.4ResNet-34
Out-of-Distribution DetectionCIFAR-10 vs iSUNAUROC100DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-10 vs iSUNAUROC100ResNet-34
Out-of-Distribution DetectionCIFAR-100 vs SVHNAUROC98.4DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-100 vs SVHNAUROC97.9ResNet-34
Out-of-Distribution DetectionCIFAR-100 vs GaussianAUROC100DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-100 vs GaussianAUROC100ResNet-34
Out-of-Distribution DetectionCIFAR-10 vs LSUN (R)AUROC100ResNet-34
Out-of-Distribution DetectionCIFAR-10 vs LSUN (R)AUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs UniformAUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs UniformAUROC100ResNet-34
Out-of-Distribution DetectionSVHN vs ImageNet (R)AUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs ImageNet (R)AUROC100ResNet-34
Out-of-Distribution DetectionCIFAR-10 vs ImageNet (C)AUROC99.9DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-10 vs ImageNet (C)AUROC99.8ResNet-34
Out-of-Distribution DetectionCIFAR-100 vs LSUN (R)AUROC99.7DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-100 vs LSUN (R)AUROC99.6ResNet-34
Out-of-Distribution DetectionSVHN vs GaussianAUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs GaussianAUROC100ResNet-34
Out-of-Distribution DetectionSVHN vs CIFAR-100AUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs CIFAR-100AUROC99.8ResNet-34
Out-of-Distribution DetectionSVHN vs CIFAR-10AUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs CIFAR-10AUROC99.8ResNet-34
Out-of-Distribution DetectionCIFAR-10 vs SVHNAUROC100DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-10 vs SVHNAUROC99.8ResNet-34
Out-of-Distribution DetectionCIFAR-100 vs UniformAUROC100DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-100 vs UniformAUROC100ResNet-34
Out-of-Distribution DetectionCIFAR-10 vs ImageNet (R)AUROC99.9DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-10 vs ImageNet (R)AUROC99.9ResNet-34
Out-of-Distribution DetectionSVHN vs ImageNet (C)AUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs ImageNet (C)AUROC100ResNet-34
Out-of-Distribution DetectionSVHN vs LSUN (R)AUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs LSUN (R)AUROC100ResNet-34
Out-of-Distribution DetectionCIFAR-100 vs iSUNAUROC99.5DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-100 vs iSUNAUROC99.3ResNet-34
Out-of-Distribution DetectionCIFAR-100 vs LSUN (C)AUROC97.8ResNet-34
Out-of-Distribution DetectionCIFAR-100 vs LSUN (C)AUROC96.1DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-100 vs ImageNet (R)AUROC99.5DenseNet-BC-100
Out-of-Distribution DetectionCIFAR-100 vs ImageNet (R)AUROC99.2ResNet-34
Out-of-Distribution DetectionSVHN vs LSUN (C)AUROC100DenseNet-BC-100
Out-of-Distribution DetectionSVHN vs LSUN (C)AUROC99.9ResNet-34

Related Papers

ZClassifier: Temperature Tuning and Manifold Approximation via KL Divergence on Logit Space2025-07-14Safe Domain Randomization via Uncertainty-Aware Out-of-Distribution Detection and Policy Adaptation2025-07-08FA: Forced Prompt Learning of Vision-Language Models for Out-of-Distribution Detection2025-07-06Out-of-distribution detection in 3D applications: a review2025-07-01Generative Adversarial Evasion and Out-of-Distribution Detection for UAV Cyber-Attacks2025-06-26Enclosing Prototypical Variational Autoencoder for Explainable Out-of-Distribution Detection2025-06-17A Variational Information Theoretic Approach to Out-of-Distribution Detection2025-06-17FindMeIfYouCan: Bringing Open Set metrics to $\textit{near} $, $ \textit{far} $ and $\textit{farther}$ Out-of-Distribution Object Detection2025-06-16