Research map / Label-efficient and robust learning / Recognition and long-tail
Distribution Alignment Optimization through Neural Collapse for Long-tailed Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DisA, an optimal-transport distribution-alignment regularizer that pulls last-layer features toward a balanced simplex ETF distribution to induce neural collapse for long-tailed classification, combinable with existing long-tailed losses.
Problem
Neural collapse is hard to achieve when training on long-tailed data, so minority-class representations become entangled and test performance degrades.
Approach
Treat last-layer features of the imbalanced training batch as an empirical distribution and a simplex ETF (balanced by construction) as a target distribution; minimize an entropic-regularized optimal transport cost (cosine distance, Sinkhorn) between them as a plug-and-play regularizer, usable with fixed ETF classifiers or learnable classifiers.
Key results
- On CIFAR-LT-10 with ρ=200, ETF-DR+DisA improves over ETF-DR from 71.9% to 73.7%, and CE-DRW+DisA reaches 77.7% vs. 75.1% for CE-DRW.
- On CIFAR-LT-100 with ρ=10, INC-DRW-cRT+DisA reaches 65.9% vs. 63.6% for INC-DRW-cRT.
- On ImageNet-LT with ResNeXt-50, INC-DRW-cRT+DisA reaches 54.5% overall vs. 54.2% for INC-DRW-cRT; RBL+DisA reaches 53.5% vs. 53.3% for RBL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DisA to CE | CE with ResNet-32 · CIFAR-LT-10 (ρ=200) | top-1 test accuracy (%): 67.3 -> 69.2 | Helped |
| add DisA to ETF-DR | ETF-DR with ResNet-32 · CIFAR-LT-10 (ρ=200) | top-1 test accuracy (%): 71.9 -> 73.7 | Helped |
| add DisA to RBL | RBL with ResNet-32 · CIFAR-LT-10 (ρ=200) | top-1 test accuracy (%): 81.2 -> 82.0 | Helped |
| add DisA to INC-DRW-cRT | INC-DRW-cRT with ResNet-32 · CIFAR-LT-10 (ρ=100) | top-1 test accuracy (%): 82.6 -> 82.8 | Helped |
| add DisA to RBL | RBL with ResNet-32 · CIFAR-LT-100 (ρ=200) | top-1 test accuracy (%): 48.9 -> 48.9 | No clear effect |
| add DisA to CE-DRW | CE-DRW with ResNet-32 · CIFAR-LT-100 (ρ=10) | top-1 test accuracy (%): 61.5 -> 65.7 | Helped |
| add DisA to CE-DRW | CE-DRW with ResNet-50 · ImageNet-LT | top-1 test accuracy (%): 47.1 -> 53.2 | Helped |
| add DisA to CE | CE with ResNet-50 · ImageNet-LT Many-shot classes | top-1 test accuracy (%): 68.4 -> 67.7 | Hurt |
| add DisA to CE | CE with ResNet-50 · ImageNet-LT Few-shot classes | top-1 test accuracy (%): 4.8 -> 7.3 | Helped |
| add DisA to INC-DRW-cRT | INC-DRW-cRT with ResNeXt-50 · ImageNet-LT (All) | top-1 test accuracy (%): 54.2 -> 54.5 | Helped |
| add DisA to INC-DRW | INC-DRW with ResNeXt-50 · ImageNet-LT (All) | top-1 test accuracy (%): 53.6 -> 53.9 | Helped |
| add DisA to RBL | RBL with ResNeXt-50 · ImageNet-LT (All) | top-1 test accuracy (%): 53.3 -> 53.5 | Helped |
Surprising findings
- The paper reports that DisA on its own does not beat RBL on CIFAR-LT-10 and that RBL+DisA is unsatisfactory relative to the originally reported RBL numbers, attributed to reproduction discrepancy.
Limitations
- NC's dimension constraint (d ≥ K) makes ETF alignment hard for high-dimensional models or datasets with many classes.
- Gains on CIFAR-LT and ImageNet-LT are often small (e.g., under 1 point for several combinations).
Experimental setup
Datasets: CIFAR-LT-10 CIFAR-LT-100 ImageNet-LT
Benchmarks: CIFAR-LT-10 CIFAR-LT-100 ImageNet-LT
Metrics: top-1 accuracy many/medium/few-shot accuracy covariance traces
Baselines: CE CE-DRW LDAM-DRW INC-DRW INC-DRW-cRT ETF-CE ETF-DR RBL ARB MiSLAS RIDE TSC
Builds on: neural collapse Dot-Regression loss Inducing Neural Collapse (INC) Representation-Balanced Learning (RBL) Sinkhorn algorithm
Closest papers by meaning
- Neural Collapse To Multiple Centers For Imbalanced Data (NeurIPS 2024 · similarity 0.87)
- Inverse Weight-Balancing for Deep Long-Tailed Learning (AAAI 2024 · similarity 0.87)
- Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Features Model (ICML 2024 · similarity 0.86)
- Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective (NeurIPS 2024 · similarity 0.85)
- Distilling Long-tailed Datasets (CVPR 2025 · similarity 0.85)
- Long-Tailed Learning as Multi-Objective Optimization (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.