Research map / Label-efficient and robust learning / OOD and anomaly detection
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces BROAD, a five-shift-type OOD detection benchmark on ImageNet-1K, shows existing post-hoc detectors are inconsistent across shift types, and proposes a Gaussian-mixture ensemble over existing detection scores that improves average broad detection AUC.
Problem
Existing OOD detection research focuses almost exclusively on novel-class shifts, so detectors may be overspecialized and unreliable on other anomalous inputs such as adversarial attacks, synthetic images, corruptions, and multi-label images.
Approach
Build a broad benchmark (BROAD) spanning novel classes, adversarial perturbations, synthetic images, corruptions, and multi-label images relative to ImageNet-1K, and learn a Gaussian mixture over a selected set of existing post-hoc detection scores to form an ensemble that performs consistently across shift types without using OOD samples.
Key results
- Ens-V reaches average DSD AUC of 84.42 on ViT, a 6.86% improvement over the best baseline reported in the text; Ens-R reaches 73.87 average on ResNet-50.
- Existing baselines are inconsistent across shift types; most are Pareto-optimal, with strengths on some shift types and weaknesses on others.
- Ens-F adds only about 19-25% inference overhead over a forward pass, while Ens-V with ResNet-50 costs up to 13.92x.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Gaussian mixture ensemble of detection scores (Ens-V, ViT) | OOD detection with ViT · BROAD DSD, averaged over distribution shift types | AUC: +6.86% over best baseline | Helped |
| Gaussian mixture ensemble of detection scores (Ens-R, ResNet-50) | OOD detection with ResNet-50 · BROAD DSD, averaged over distribution shift types | AUC: +4.04% over best baseline | Helped |
| Fast 5-score Gaussian mixture ensemble (Ens-F) | OOD detection · BROAD DSD, averaged over distribution shift types | AUC | Mixed |
| Fast 5-score Gaussian mixture ensemble (Ens-F) | Error detection · ViT and ResNet-50 error detection average | AUC | Helped |
| Existing detectors evaluated per shift type | Baselines (e.g., MDS_all, Dice, ODIN) · BROAD DSD across shift types | AUC | Mixed |
| Mahalanobis distance on all layers (MDS_all) versus last layer | ViT multi-label detection · Multi-labels DSD | AUC: 26.06 vs 63.41 for MDS_l | Hurt |
| GMM component count n=10 vs other values (ablation) | Ens-V and Ens-R · Average OOD detection AUC | AUC: n=1 to 20 give 71.44 to 73.51 for ResNet-50 | No clear effect |
| Ensemble of MSP, max logits, MDS and EBO (Ens-F) vs single baselines in error detection | ResNet-50 error detection on in-distribution · In-distribution error detection AUC | AUC | Mixed |
Surprising findings
- Methods that excel on novel-class detection (e.g., MDS_l on ViT) can be near random or worse on multi-label or adversarial shifts, with MDS_all dropping to 26.06 AUC on multi-label detection for ViT.
- Detection performance of CoComageNet-mono is near random for all baselines, indicating CoComageNet detection is driven by the presence of multiple labels rather than dataset differences.
Limitations
- Uses ImageNet-1K as the only in-distribution dataset, limiting coverage to vision classification.
- Relies on default ODIN hyperparameters tuned on novel-class OOD detection, which may bias results.
- Ensemble variants Ens-V and Ens-R have high computational overhead, with Ens-F as a cheaper alternative.
Experimental setup
Datasets: ImageNet-1K iNaturalist ImageNet-O OpenImage-O ImageNet-C CoCo 2017 BigGAN-generated ImageNet-like images Stable Diffusion 2 generated images
Benchmarks: BROAD
Metrics: AUC error detection AUC
Baselines: MSP ODIN ViM Mahalanobis distance (MDS) ReAct GradNorm EBO ASH SHE DOCTOR CADet Dice
Builds on: OpenOOD MSP ViM Mahalanobis distance ODIN
Closest papers by meaning
- Out-Of-Distribution Detection with Diversification (Provably) (NeurIPS 2024 · similarity 0.92)
- DOS: Diverse Outlier Sampling for Out-of-Distribution Detection (ICLR 2024 · similarity 0.90)
- Out-of-Distribution Detection with Prototypical Outlier Proxy (AAAI 2025 · similarity 0.90)
- Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection (NeurIPS 2025 · similarity 0.90)
- ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms (ICLR 2024 · similarity 0.90)
- Position: Supervised Classifiers Answer the Wrong Questions for OOD Detection (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.