Research map / Learning theory and algorithms / Kernels and regression theory
Perturbation Analysis of Neural Collapse
Tom Tirer, Haoxiang Huang, Jonathan Niles-Weed
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase weight decay in feature mapping layers (vs. baseline 5e-4) (ablation) | ResNet18 on CIFAR-10 (MSE no bias and CE with bias) · CIFAR-10 ResNet18 NC metrics | distance from exact NC (NC1, NC2) | Improved |
| Zero weight decay in feature mapping layers (ablation) | ResNet18 on CIFAR-10 · CIFAR-10 ResNet18 NC metrics | distance from exact NC (NC1, NC2) | Hurt |
| Modify weight decay in feature mapping vs. last layer (ablation) | ResNet18 on CIFAR-10 · CIFAR-10 ResNet18 NC metrics | deviation from baseline NC curves | Mixed |
| Depthwise layer-wise addition of hidden layers (features from previous layers fixed) | MLP on CIFAR-10 · CIFAR-10 layer-wise MLP | NC1 metrics of deepest features | Improved |
| Gradient flow on central path of UFM (no added assumptions/renormalization) | Unconstrained features model with MSE · theory | tilde NC1 = Tr(Sigma_W)/Tr(Sigma_B) | Improved |
| Increase lambda_W (last-layer regularization) vs lambda_H | Near-collapse perturbation analysis · theory | singular values of intra-class blocks F_kk | No clear effect |
Closest papers by meaning
- Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse (ICLR 2025 · similarity 0.90)
- Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data (NeurIPS 2025 · similarity 0.89)
- Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model (NeurIPS 2025 · similarity 0.89)
- Generalized Neural Collapse for a Large Number of Classes (ICML 2024 · similarity 0.89)
- Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.88)
- Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
