Research map / Label-efficient and robust learning / OOD and anomaly detection
How to Overcome Curse-of-Dimensionality for Out-of-Distribution Detection?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Subspace Nearest Neighbor (SNN), which trains a classifier with class-dependent top-s feature subspace masking so that k-NN distance OOD detection on the learned features avoids the curse of dimensionality.
Problem
Distance-based OOD detection (e.g., k-NN on penultimate features) degrades in high-dimensional feature spaces due to the curse of dimensionality.
Approach
During training, for each class the logit is computed from only the top-s most relevant feature dimensions (sum of the top-s entries of w_c ⊙ h(x)), a training-time subspace regularizer; k-NN distance is then computed on the subspace-regularized normalized features, with a theoretical bound linking embedding dimension to density-estimation error.
Key results
- On CIFAR-100 (DenseNet-101), SNN reduces average FPR95 to 31.25% vs. 47.21% for KNN (about 15.96 points), with AUROC 90.76 and ID accuracy 75.59%.
- On CIFAR-10 far-OOD, SNN achieves average FPR95 of 14.99% and AUROC 97.15, versus 18.50% for KNN.
- Training with SNN takes 1.75 h vs. 2.10 h for standard training on CIFAR-10 (ResNet-101 setting in table caption).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with class-relevant feature subspaces instead of full-space KNN | KNN OOD detection · CIFAR-100 (six OOD datasets average) | FPR95: 47.21 -> 31.25 | Helped |
| Train with class-relevant feature subspaces instead of full-space KNN | KNN OOD detection · CIFAR-10 near-OOD (CIFAR-100 OOD) | FPR95: 58.34 -> 50.10 | Helped |
| Train with class-relevant feature subspaces and use Mahalanobis distance | Mahalanobis OOD detection · CIFAR-10 | FPR95: 44.55 -> 34.68 | Helped |
| Train with class-relevant feature subspaces and use Mahalanobis distance | Mahalanobis OOD detection · CIFAR-100 | FPR95: 56.93 -> 55.05 | Helped |
| Replace selecting the most relevant dimensions with a random subspace (ablation) | SNN · CIFAR-100 (six OOD datasets average) | FPR95: 31.25 -> 42.21 | Hurt |
| Select the least relevant feature dimensions instead of the most relevant (ablation) | SNN · CIFAR-100 (six OOD datasets average) | FPR95: 31.25 -> 63.96 | Hurt |
| Use a very small relevance ratio (r=0.05) (ablation) | SNN · CIFAR-100 | OOD detection performance | Hurt |
| Use a mild relevance ratio such as r=0.25 (ablation) | SNN · CIFAR-100 | OOD detection performance | Helped |
| Vary the number of nearest neighbors k within mild values (ablation) | SNN · CIFAR-100 | OOD detection performance | No clear effect |
Surprising findings
- Post-hoc sparsification (DICE) substantially reduces ID accuracy (68.65% on CIFAR-100), whereas training-time subspace learning preserves it.
- Selecting the least relevant dimensions yields much worse OOD performance (FPR95 63.96) than random subsets (42.21) or most relevant (31.25).
Limitations
- Performance depends on hyperparameters r and k, selected via a validation strategy using Gaussian noise images rather than real OOD data.
- Theoretical bound is an upper-bound-style argument that relies on density estimation approximations and is not tightly tied to empirical behavior.
- Evaluated mainly on small-resolution CIFAR benchmarks with DenseNet-101; ImageNet-100 experiments are limited.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-100 Textures SVHN LSUN-Crop LSUN-Resize iSUN Places365 iNaturalist SUN Gaussian noise validation images
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-100 OOD benchmarks
Metrics: FPR95 AUROC ID accuracy SCE ECE training time
Baselines: MSP ODIN Generalized-ODIN Energy score ReAct GradNorm LogitNorm DICE Mahalanobis KNN Unit Dropout Adaptive Dropout
Builds on: KNN OOD detection DICE Mahalanobis distance Targeted Dropout Adaptive Dropout
Closest papers by meaning
- Learning to Shape In-distribution Feature Space for Out-of-distribution Detection (NeurIPS 2024 · similarity 0.88)
- EAT: Towards Long-Tailed Out-of-Distribution Detection (AAAI 2024 · similarity 0.87)
- Mining In-distribution Attributes in Outliers for Out-of-distribution Detection (AAAI 2025 · similarity 0.86)
- Learning with Mixture of Prototypes for Out-of-Distribution Detection (ICLR 2024 · similarity 0.86)
- Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.86)
- Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.