Research map / Label-efficient and robust learning / OOD and anomaly detection
Learning to Shape In-distribution Feature Space for Out-of-distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Distributional Representation Learning (DRL), which trains a classifier so its penultimate features follow a pre-defined vMF-style mixture distribution, using an online Bessel-function approximation and an EM formulation, for density-based OOD detection.
Problem
Post-hoc OOD detection relies on distributional assumptions about the learned feature space (e.g., Gaussian), which often do not hold, creating a mismatch between assumed and learned distributions.
Approach
Explicitly shape the ID feature space during pre-training to follow a mixture distribution (vMF-like with unnormalized prototypes), approximate the intractable normalization constant online via an asymptotic Bessel expansion, and train with an EM-derived ELBO plus classification objective, with half of each mini-batch reused from the previous batch for consistent soft labels.
Key results
- On CIFAR-10 (ResNet-18), DRL achieves 11.58% average FPR95 and 97.83% AUROC, outperforming PALM (14.96% FPR95) by 3.38 points.
- On CIFAR-100 (ResNet-34), DRL reports 35.66% average FPR95, behind PALM (28.02%) on average.
- On hard OOD detection, DRL achieves 60.88% average FPR95 vs. 64.40% for PALM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DRL instead of PALM | CIFAR-10 OOD detection with ResNet-18 · CIFAR-10 OOD, average | FPR95: 14.96 -> 11.58 | Helped |
| Use DRL instead of PALM | CIFAR-10 OOD detection with ResNet-18 · CIFAR-10 OOD, average | AUROC: 97.57 -> 97.83 | Helped |
| Use DRL instead of PALM | CIFAR-100 OOD detection with ResNet-34 · CIFAR-100 OOD, average | FPR95: 28.02 -> 35.66 | Hurt |
| Use DRL instead of PALM | CIFAR-100 OOD detection with ResNet-34 · CIFAR-100 OOD, average | AUROC: 93.82 -> 90.13 | Hurt |
| Use DRL instead of PALM | Hard OOD detection with ResNet-34 · Hard OOD, average | FPR95: 64.40 -> 60.88 | Helped |
| Use DRL instead of PALM | Hard OOD detection with ResNet-34 · Hard OOD, average | AUROC: 81.76 -> 84.50 | Helped |
| Use DRL instead of PALM | Unsupervised CIFAR-100 OOD detection · Unsupervised CIFAR-100 OOD, average | FPR95: 43.37 -> 40.66 | Helped |
| Use DRL instead of PALM | Unsupervised CIFAR-100 OOD detection · Unsupervised CIFAR-100 OOD, average | AUROC: 88.07 -> 89.00 | Helped |
| Use ℓ2-normalized rather than unnormalized class prototypes (ablation) | DRL · OOD detection | OOD detection performance | Hurt |
| Remove sequential sampling rearrangement (ablation) | DRL · OOD detection | OOD detection performance | Hurt |
Surprising findings
- On CIFAR-100, DRL's average FPR95 (35.66%) is worse than PALM's (28.02%), contrary to the paper's general claim of state-of-the-art performance.
- Unnormalized class prototypes give a significant OOD performance gain over l2-normalized prototypes.
Limitations
- Only one realization of the non-negative density function is explored (stated in the appendix).
- Code is not released at submission time; release promised upon acceptance.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-100 ImageNet-1K
Benchmarks: CIFAR-10 OOD (SVHN, Places365, LSUN, iSUN, Texture) CIFAR-100 OOD hard OOD (LSUN-Fix, ImageNet-Fix, ImageNet-Resize, CIFAR-10) unsupervised OOD on CIFAR-100 ImageNet-100 OOD (iNaturalist, SUN, Places365, Textures)
Metrics: FPR95 AUROC
Baselines: MSP ODIN Energy ReAct ASH Mahalanobis KNN CONJ Vim VOS CSI SSD+
Builds on: CIDER PALM Mahalanobis vMF mixture models ELBO / EM
Closest papers by meaning
- Learning with Mixture of Prototypes for Out-of-Distribution Detection (ICLR 2024 · similarity 0.91)
- Provable Discriminative Hyperspherical Embedding for Out-of-Distribution Detection (AAAI 2025 · similarity 0.90)
- Fast Decision Boundary based Out-of-Distribution Detector (ICML 2024 · similarity 0.90)
- Exploiting Discrepancy in Feature Statistic for Out-of-Distribution Detection (AAAI 2024 · similarity 0.90)
- Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.89)
- Discriminability-Driven Channel Selection for Out-of-Distribution Detection (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.