AI Paper Atlas

Research map / Optimization / Implicit bias and generalization

Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model

Peter Súkeník, Marco Mondelli, Christoph H Lampert

NeurIPS 2023 · Optimization › Implicit bias and generalization · approach: Neural tangent kernel analysis · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train the DUFM with gradient descent (L=3 and L=6, n=50, width 64) instead of solving analyticallyDeep unconstrained features model · DUFM synthetic trainingDNC1 within-class variabilityImproved
measure DNC2 on the second layer features H_2 and first layer H_1DUFM trained by gradient descent · DUFM synthetic trainingDNC2 orthogonalityNo clear effect
measure DNC3 alignment on all layers including the firstDUFM trained by gradient descent · DUFM synthetic trainingDNC3 alignment (lower better)Improved
compare final training loss of gradient descent against numerically computed global optimumDUFM · DUFM synthetic trainingtraining lossNo clear effect
replace unconstrained features with output of random ResNet20 and train end-to-end with frozen pre-trained DUFM headResNet20 + DUFM head · CIFAR-10 classes 0 and 1DNC metricsImproved
use smaller learning rates than 0.5 in DUFM training (ablation)DUFM · DUFM synthetic trainingDNC convergenceNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy