Research map / Label-efficient and robust learning / Recognition and long-tail
Neural Collapse To Multiple Centers For Imbalanced Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neural Collapse to Multiple Centers (NCMC) under an MSE-type UFM analysis and a fixed-classifier Cosine Loss with a class-aware center-count strategy, achieving long-tail accuracy comparable to classical imbalanced learning methods.
Problem
Neural collapse with fixed classifiers fails to match classical imbalanced learning methods with learnable classifiers; the relation between the optimal NC structure and the classification rule under imbalance is unclear.
Approach
Argues that minority classes should align with more orthogonal directions (centers) to improve classification of hard-to-predict features, defines a Generalized Classification Rule (GCR) and shows that under NC to multiple centers the regular nearest-neighbor rule approximates GCR; proposes a Cosine Loss for a fixed multi-center frame and a class-aware strategy to set the number of centers per cla
Key results
- CAL (Cosine Loss with class-aware centers) on ResNet50 reaches 80.0% on CIFAR-10 with imbalance ratio 0.005 versus 72.3% for CE and 79.5% for ARBloss.
- On CIFAR-100 with imbalance ratio 0.005 and ResNet50 with mixup, CAL achieves 46.5% versus 42.4% for CE and 40.3% for KCL.
- On ImageNet, CAL reaches 49.7% versus 44.2% for CE (Mixup) but below KCL (51.5) and ARBloss (52.8).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CE with CAL | ResNet · CIFAR-10, τ=0.005 | accuracy (%): 72.3±0.1 -> 80.0±0.5 | Helped |
| Replace CE with CAL | ResNet · CIFAR-10, τ=0.01 | accuracy (%): 78.6±0.2 -> 84.1±0.4 | Helped |
| Replace CE with CAL | ResNet · CIFAR-100, τ=0.005 | accuracy (%): 42.4±0.5 -> 46.5±0.5 | Helped |
| Replace CE with CAL | ResNet · CIFAR-100, τ=0.01 | accuracy (%): 46.7±0.3 -> 50.1±0.2 | Helped |
| Replace CE with CAL | ResNet · SVHN, τ=0.005 | accuracy (%): 39.4±0.2 -> 43.5±0.1 | Helped |
| Replace SETF with CAL | ResNet · SVHN, τ=0.02 | accuracy (%): 49.6±0.2 -> 49.1±0.1 | Hurt |
| Replace CE with CAL | ResNet · STL-10, τ=0.005 | accuracy (%): 33.6 ±0.1 -> 40.5±0.2 | Helped |
| Replace KCL with CAL | ResNet50 · ImageNet, τ=0.1 | accuracy (%): 51.5 -> 49.7±0.2 | Hurt |
| Add Mixup training (ablation) | CAL with ResNet50 · CIFAR-100, τ=0.005 | accuracy (%): 40.6±0.3 -> 46.5±0.5 | Helped |
| Add Mixup training (ablation) | CAL with ResNet50 · CIFAR-100, τ=0.01 | accuracy (%): 44.7±0.2 -> 50.1± 0.3 | Helped |
| Add Mixup training (ablation) | CAL with ResNet50 · CIFAR-100, τ=0.02 | accuracy (%): 50.2±0.5 -> 54.3±0.4 | Helped |
| Add Mixup training (ablation) | CAL with ResNet50 · balanced CIFAR-100 | accuracy (%): 78.5±0.4 -> 79.4±0.4 | Helped |
Surprising findings
- Under balanced data the proposed method performs slightly worse than or on par with CE, and the cosine loss underperforms at high imbalance-ratio regimes where minor classes lose their randomness.
- The class-aware strategy as implemented can emphasize the middle class rather than the minority class, according to the authors' own example.
Limitations
- Theoretical motivation considers only the Gaussian case.
- Performance is weak on ImageNet and high-dimensional datasets with many classes, and only ResNet and DenseNet were tested.
- The class-aware strategy for choosing expansion factors is admittedly suboptimal.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN STL-10 ImageNet
Benchmarks: CIFAR-10-LT CIFAR-100-LT SVHN-LT STL-10-LT ImageNet-LT
Metrics: top-1 accuracy neural collapse metric
Baselines: CE SETF LDAM-DRW KCL ARBLoss RBL
Builds on: Neural Collapse Unconstrained Features Model SETF ARBLoss
Closest papers by meaning
- Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Features Model (ICML 2024 · similarity 0.90)
- Generalized Neural Collapse for a Large Number of Classes (ICML 2024 · similarity 0.89)
- Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime (ICML 2025 · similarity 0.88)
- The Prevalence of Neural Collapse in Neural Multivariate Regression (NeurIPS 2024 · similarity 0.88)
- Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse (ICLR 2025 · similarity 0.87)
- Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.