Research map / Label-efficient and robust learning / Recognition and long-tail
Navigate Beyond Shortcuts: Debiased Learning Through the Lens of Neural Collapse
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ETF-Debias, which uses a fixed, training-free simplex ETF as shortcut "prime" features tied to bias attributes (plus a prime reinforcement regularizer) so models skip shortcut learning and generalize on biased datasets.
Problem
Neural networks trained on biased datasets with imbalanced bias attributes quickly learn shortcut correlations, forming non-collapsed feature spaces that generalize poorly to bias-conflicting test samples; existing debiasing methods need costly auxiliary models or contrastive samples.
Approach
Extend Neural Collapse analysis to biased data and use a randomly initialized, fixed simplex ETF over bias-attribute categories as approximated "perfect" shortcut primes concatenated to learnable features during training, with a regularizer forcing reliance on primes; at test time primes are set to null vectors.
Key results
- ETF-Debias reaches 65.34% on Corrupted CIFAR-10 at 5.0% ratio versus 62.43% for the best baseline (DisEnt), a +2.91 gain.
- On Biased FFHQ at 5.0% ratio, ETF-Debias achieves 94.00% versus 84.80% for the best baseline, up to 6.6 points improvement.
- Ablation shows randomly initialized prime features cause severe performance degradation compared with ETF-based primes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ETF-Debias instead of the strongest baseline | Colored MNIST · bias ratios 0.5%, 1.0%, 2.0%, and 5.0% | accuracy: 71.63±0.28 vs 70.53±0.46; 81.97±0.26 vs 83.34±0.37; 86.00±0.03 vs 85.90±0.23; 91.36±0.21 vs 91.27±0.31 | Mixed |
| Use ETF-Debias instead of the strongest baseline | Corrupted CIFAR-10 · bias ratios 0.5%, 1.0%, 2.0%, and 5.0% | accuracy: 40.06±0.03 vs 36.66±0.74; 47.52±0.26 vs 45.66±1.05; 54.64±0.42 vs 50.11±0.69; 65.34±0.60 vs 62.43±0.57 | Helped |
| Use ETF-Debias instead of the strongest baseline | Biased FFHQ · bias ratios 0.5%, 1.0%, 2.0%, and 5.0% | accuracy: 73.60±1.22 vs 67.07±2.37; 76.53±1.10 vs 73.53±1.62; 85.20±0.61 vs 80.20±2.78; 94.00±0.72 vs 87.40±2.00 | Helped |
| Use ETF-Debias instead of the strongest baseline | Dogs & Cats · bias ratios 1.0% and 5.0% | accuracy: 80.07±0.90 vs 78.87±2.40; 92.18±0.62 vs 89.12±0.18 | Helped |
| Use ETF-Debias instead of the strongest baseline | BAR · bias ratios 1.0% and 5.0% | accuracy: 72.79±0.21 vs 71.70±1.33; 83.66±0.21 vs 82.00±1.24 | Helped |
| Add prime reinforcement regularization (ablation) | ETF-Debias · unbiased test sets across the evaluated datasets | accuracy | Helped |
| Replace ETF prime features with randomly initialized prime features (ablation) | ETF-Debias · five datasets at 5% bias ratio | accuracy | Hurt |
Surprising findings
- Randomly initialized prime features perform much worse than ETF primes, indicating the geometric structure matters.
Limitations
- Gains on some settings are small or negative (e.g., Colored MNIST 1.0% ratio is 1.37 below the best baseline).
- Requires pre-defined bias attribute labels for training primes, so it depends on bias supervision.
Experimental setup
Datasets: Colored MNIST Corrupted CIFAR-10 Biased FFHQ Dogs & Cats BAR CIFAR-10
Benchmarks: Colored MNIST Corrupted CIFAR-10 Biased FFHQ Dogs & Cats BAR
Metrics: accuracy on unbiased test set neural collapse metrics NC1-NC3
Baselines: LfF LfF+BE EnD SD DisEnt Selecmix
Builds on: prime training neural collapse learning from failure (LfF) Minority Collapse
Closest papers by meaning
- Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data (NeurIPS 2025 · similarity 0.87)
- On the Foundations of Shortcut Learning (ICLR 2024 · similarity 0.86)
- Generalized Neural Collapse for a Large Number of Classes (ICML 2024 · similarity 0.86)
- Neural collapse vs. low-rank bias: Is deep neural collapse really optimal? (NeurIPS 2024 · similarity 0.86)
- Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime (ICML 2025 · similarity 0.86)
- Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.