Research map / Optimization / Implicit bias and generalization
Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model
Peter Súkeník, Marco Mondelli, Christoph H Lampert
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the DUFM with gradient descent (L=3 and L=6, n=50, width 64) instead of solving analytically | Deep unconstrained features model · DUFM synthetic training | DNC1 within-class variability | Improved |
| measure DNC2 on the second layer features H_2 and first layer H_1 | DUFM trained by gradient descent · DUFM synthetic training | DNC2 orthogonality | No clear effect |
| measure DNC3 alignment on all layers including the first | DUFM trained by gradient descent · DUFM synthetic training | DNC3 alignment (lower better) | Improved |
| compare final training loss of gradient descent against numerically computed global optimum | DUFM · DUFM synthetic training | training loss | No clear effect |
| replace unconstrained features with output of random ResNet20 and train end-to-end with frozen pre-trained DUFM head | ResNet20 + DUFM head · CIFAR-10 classes 0 and 1 | DNC metrics | Improved |
| use smaller learning rates than 0.5 in DUFM training (ablation) | DUFM · DUFM synthetic training | DNC convergence | No clear effect |
Closest papers by meaning
- Neural collapse vs. low-rank bias: Is deep neural collapse really optimal? (NeurIPS 2024 · similarity 0.93)
- The Persistence of Neural Collapse Despite Low-Rank Bias (NeurIPS 2025 · similarity 0.92)
- Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data (NeurIPS 2025 · similarity 0.92)
- Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model (NeurIPS 2025 · similarity 0.91)
- Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse (ICLR 2025 · similarity 0.91)
- Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
