Research map / Label-efficient and robust learning / OOD and anomaly detection
Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Separation Loss grounded in Neural Collapse that pushes auxiliary-OOD features into the subspace orthogonal to the classifier weights, improving OOD detection after fine-tuning with auxiliary outliers.
Problem
Out-of-distribution detection with auxiliary OOD data: existing outlier-exposure style losses only enlarge ID-OOD separation in the output space, and the diversity of OOD features makes it hard to design direct feature-space separation losses.
Approach
Leverage Neural Collapse (ID penultimate features align with class weights) so that OOD features are penalized for cosine similarity with the FC weight vectors, pushing them into orthogonal dimensions; combined with a clustering loss on ID features and a score that adds MSP and the feature-weight cosine term.
Key results
- On CIFAR100 (WRN-40-2), average FPR95 of 29.47% and AUROC 94.00%, versus DAL 31.47%/92.82% and OE 37.77%/92.21%.
- On CIFAR10 (WRN-40-2), average FPR95 2.49% vs OE 3.36%, with AUROC 98.93%.
- On ImageNet-1k with ResNet50, average FPR95 56.73% and AUROC 84.62%, versus DAL 59.39%/84.04% and OE 59.66%/83.89%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Separation and Cluster losses to OE fine-tuning (ablation) | WideResNet-40-2 · CIFAR10, average across five OOD datasets | FPR95: 3.36 -> 2.49 | Helped |
| Add Separation and Cluster losses to OE fine-tuning (ablation) | WideResNet-40-2 · CIFAR100, average across five OOD datasets | FPR95: 37.77 -> 29.47 | Helped |
| Add Separation loss to OE fine-tuning without Cluster loss (ablation) | WideResNet-40-2 · CIFAR100, average across five OOD datasets | FPR95: 37.77 -> 33.30 | Helped |
| Add Cluster loss to OE fine-tuning without Separation loss (ablation) | WideResNet-40-2 · CIFAR100, average across five OOD datasets | FPR95: 37.77 -> 39.91 | Hurt |
| Replace cosine-similarity separation with Euclidean-distance loss (ablation) | WideResNet-40-2 · CIFAR100, average across five OOD datasets | FPR95: 29.47 -> 43.20 | Hurt |
| Replace the proposed score function with MSP (ablation) | Ours · CIFAR100, average across five OOD datasets | AUROC: 94.00 -> 93.27 | Hurt |
Surprising findings
- Euclidean-distance-based separation loss performs clearly worse than the cosine-based orthogonality loss on CIFAR100 (avg FPR95 43.20 vs 29.47).
- Adding the clustering loss alone worsens performance (OE+Clu FPR95 39.91 vs OE 37.77 on CIFAR100), but combined with separation it gives the best result.
Limitations
- Requires feature dimension to exceed the number of classes; performance gains are smaller when features are low-dimensional (WRN-40-1).
- Relies on the Neural Collapse property, which may fail under class imbalance (Minority Collapse), limiting applicability.
- Results on some datasets (e.g., CIFAR10 Places365, DenseNet121 on CIFAR100) are not uniformly best, and variance across runs is reported as nontrivial for baselines.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet-1k 80 Million Tiny Images ImageNet-21k-p SVHN LSUN iSUN Textures Places365 iNaturalist SUN
Benchmarks: CIFAR10 CIFAR100 ImageNet-1k Hard OOD (LSUN-Fix, ImageNet-Resize, CIFAR100, Tiny-ImageNet)
Metrics: FPR95 AUROC ID accuracy
Baselines: MSP Energy Maha KNN CSI CIDER KNN+ OE Energy-OE POEM DAL
Builds on: Outlier Exposure Energy-OE Neural Collapse DAL POEM
Closest papers by meaning
- Detecting Out-of-Distribution Through the Lens of Neural Collapse (CVPR 2025 · similarity 0.92)
- CASE: Exploiting Intra-class Compactness and Inter-class Separability of Feature Embeddings for Out-of-Distribution Detection (AAAI 2024 · similarity 0.90)
- NECO: NEural Collapse Based Out-of-distribution detection (ICLR 2024 · similarity 0.90)
- Splitting & Integrating: Out-of-Distribution Detection via Adversarial Gradient Attribution (ICML 2025 · similarity 0.89)
- MahaVar: OOD Detection via Class-wise Mahalanobis Distance Variance under Neural Collapse (NeurIPS 2026 · similarity 0.89)
- A Variational Information Theoretic Approach to Out-of-Distribution Detection (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.