Research map / Label-efficient and robust learning / OOD and anomaly detection
Plugin estimators for selective classification with out-of-distribution detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formulates selective classification with OOD detection (SCOD), derives its Bayes-optimal sample-dependent confidence-thresholding rule, and proposes black-box and loss-based plug-in estimators that generalise SC and OOD baselines.
Problem
Selective classification with out-of-distribution detection (SCOD): learning a classifier that abstains on both misclassified/hard in-distribution samples and OOD samples, which existing SC and OOD methods handle only heuristically or incompletely.
Approach
Derive the Bayes-optimal SCOD rejector, which thresholds the maximum inlier class probability against the ID-OOD density ratio, and build two plug-in estimators: a black-box combination of any off-the-shelf SC and OOD scores (ID data only), and decoupled/coupled losses trained on ID plus a wild ID-OOD mixture.
Key results
- On CIFAR-100 with a 10% OOD wild set from Random300K, the best plug-in estimator reaches AUC-RC 0.204 on SVHN versus 0.318 for MSP (lower is better).
- When the wild OOD set matches the test OOD distribution, the loss-based plug-in estimator often gives the lowest AUC-RC (e.g., 0.199 on Places vs 0.232 for MaxLogit's counterpart in Table 2 setting).
- On ImageNet with ID-only training, the black-box plug-in with residual scorer yields the lowest AUC-RC on 5 of 8 OOD datasets (e.g., 0.145 on CelebA vs 0.241 for MSP).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the residual-score black-box plug-in estimator instead of the SIRC residual-score baseline | ImageNet ID-only SCOD · ImageNet OOD datasets | AUC-RC: 5/8 | Mixed |
| Train the loss-based plug-in estimator with OOD samples when training and test OOD distributions match | CIFAR-100 SCOD · CIFAR-100 OOD benchmarks | AUC-RC | Helped |
| Use black-box plug-in estimators rather than the loss-based plug-in estimator when training and test OOD distributions differ | CIFAR-100 SCOD · CIFAR-100 OOD benchmarks | AUC-RC | Helped |
Surprising findings
- The MSP baseline can be arbitrarily bad for SCOD; the Bayes-optimal rule rejects samples with high rather than low maximum probability in the open-set case.
- When the wild OOD set differs from the test OOD set, the black-box estimators often beat the loss-based estimator that uses wild data.
Limitations
- Black-box estimator performance depends on the quality of the grad-norm OOD scorer, which is weak on some OOD datasets such as ImageNet-O.
- Loss-based estimator degrades under mismatch between training and test OOD distributions.
Experimental setup
Datasets: CIFAR-100 ImageNet 300K Random Images OpenImages SVHN Places365 LSUN Texture CelebA
Benchmarks: CIFAR-100 with SVHN, Places, LSUN, LSUN-R, Texture, OpenImages, CelebA as OOD ImageNet with Places, LSUN, CelebA, Colorectal, iNaturalist-O, Texture, OpenImages-O, ImageNet-O as OOD
Metrics: AUC-RC ID accuracy OOD precision and recall ROC-AUC FPR@95TPR
Baselines: MSP MaxLogit Energy SIRC CCE DCE Outlier Exposure (OE)
Builds on: SIRC Chow's rule outlier exposure gradient norm OOD score
Closest papers by meaning
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.89)
- Enhancing the Power of OOD Detection via Sample-Aware Model Selection (CVPR 2024 · similarity 0.88)
- Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the Role of Model Complexity (NeurIPS 2025 · similarity 0.88)
- Out-of-Distribution Detection with Prototypical Outlier Proxy (AAAI 2025 · similarity 0.88)
- What If the Input is Expanded in OOD Detection? (NeurIPS 2024 · similarity 0.88)
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.