Research map / Label-efficient and robust learning / OOD and anomaly detection
Controlling Neural Collapse Enhances Out-of-Distribution Detection and Transfer Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that neural collapse (NC) in network layers trades off OOD detection against OOD generalization, and proposes entropy regularization in the encoder (to mitigate NC for transfer) plus a frozen simplex ETF projector (to induce NC for detection).
Problem
OOD detection and OOD generalization (semantic OOD, no auxiliary OOD data) are studied separately, and a single feature space appears unable to serve both; the relationship between neural collapse and these two objectives is unclear.
Approach
Stronger NC in a layer improves OOD detection but hurts transfer, so decouple the two: an encoder regularized by a nearest-neighbor entropy loss to mitigate NC for transfer, and a frozen simplex-ETF projector with L2 normalization that induces NC for detection, with detection and ID classification using the projector and transfer using the encoder.
Key results
- Projector (stronger NC) reaches 65.10% average FPR95 detection error vs 87.62% for encoder on VGG17, and encoder reaches 41.85% transfer error vs 66.36% for projector.
- Compared with NECO on average detection error, the method reduces it from 77.82% to 65.10% for VGG17 and from 88.13% to 69.70% for ResNet18.
- Group normalization instead of batch normalization improves OOD generalization by 10.11% absolute and detection by 4.37% absolute on VGG17.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add entropy regularization to the encoder (ablation) | VGG17 · ImageNet-100; 8 OOD datasets average | OOD generalization error (%): 44.56 -> 41.85 | Helped |
| Add entropy regularization to the encoder (ablation) | VGG17 · ImageNet-100; 8 OOD datasets average | OOD detection error (%): 67.46 -> 65.10 | Helped |
| Use a fixed simplex ETF projector instead of a trainable projector (ablation) | VGG17 · ImageNet-100; 8 OOD datasets average | OOD detection error (%): 74.00 -> 65.10 | Helped |
| Add L2 normalization to the projector output (ablation) | VGG17 with fixed ETF projector · ImageNet-100; 8 OOD datasets average | OOD detection error (%): 68.93 -> 65.10 | Helped |
| Use projector depth 2 instead of depth 1 (ablation) | VGG17 with ETF projector · ImageNet-100; 8 OOD datasets average | OOD detection error (%): 87.37 -> 65.10 | Helped |
| Replace batch normalization with group normalization and weight standardization (ablation) | VGG17 encoder · ImageNet-100; 8 OOD datasets average | OOD generalization error (%): 51.96 -> 41.85 | Helped |
| Replace batch normalization with group normalization and weight standardization (ablation) | VGG17 encoder · ImageNet-100; 8 OOD datasets average | OOD detection error (%): 69.47 -> 65.10 | Helped |
| Use the proposed entropy-regularized ETF method | VGG17 trained with SGD · ImageNet-100; 8 OOD datasets average | OOD generalization error (%): 57.17 -> 50.91 | Helped |
| Use CE instead of MSE loss (ablation) | VGG17 encoder · ImageNet-100; 8 OOD datasets average | OOD generalization error (%): 59.56 -> 41.85 | Helped |
| Use CE instead of MSE loss (ablation) | VGG17 projector · ImageNet-100; 8 OOD datasets average | OOD detection error (%): 71.84 -> 65.10 | Helped |
Surprising findings
- Batch normalization induces stronger NC in the encoder and impairs OOD transfer, so replacing BN with GN+WS matters for transfer.
- CE outperforms MSE not only in generalization but also in OOD detection, which prior work did not examine.
Limitations
- Theoretical link between NC and OOD detection/generalization is largely empirical; the entropy proposition is a sketch relying on a Dirac-limit argument.
- Evaluated only on vision classification with ImageNet-100 as ID; not tested on text, audio, or other tasks.
- Baselines are not directly comparable to prior works due to a different problem setup, so comparisons are limited to in-house baselines and NECO.
Experimental setup
Datasets: ImageNet-100
Benchmarks: NINCO ImageNet-R CIFAR-100 Oxford 102 Flowers CUB-200 FGVC Aircraft Oxford-IIIT Pets STL-10
Metrics: FPR95 (OOD detection error) linear probe transfer error ID error NC1-NC4
Baselines: NECO Plastic projector Batch normalization variant MSE loss variant No entropy regularization
Builds on: neural collapse NECO energy-based OOD detection Tunnel Effect Hypothesis
Closest papers by meaning
- Detecting Out-of-Distribution Through the Lens of Neural Collapse (CVPR 2025 · similarity 0.91)
- What Variables Affect Out-of-Distribution Generalization in Pretrained Models? (NeurIPS 2024 · similarity 0.90)
- NECO: NEural Collapse Based Out-of-distribution detection (ICLR 2024 · similarity 0.90)
- Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection (ICLR 2025 · similarity 0.88)
- Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection (NeurIPS 2025 · similarity 0.87)
- Flatness is Necessary, Neural Collapse is Not: Rethinking Generalization via Grokking (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.