Research map / Optimization / Implicit bias and generalization
Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping
Xuran Meng, Jeff Yao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add spectral-criterion early stopping (FC2) | NN1 · Gaussian D1 TP=0.15 | test accuracy: 20.17% -> 25.84% | Improved |
| add spectral-criterion early stopping (FC2) | NN1 · Gaussian D1 TP=0.6 | test accuracy: 88.30% -> 88.52% | No clear effect |
| add spectral-criterion early stopping (FC1) | NN2 · Gaussian D1 TP=0.13 | test accuracy: 87.50% -> 87.03% | No clear effect |
| add spectral-criterion early stopping (FC2) | LeNet · CIFAR10 batch 256 | test accuracy: 60.49% -> 54.23% | Hurt |
| add spectral-criterion early stopping (FC1) | MiniAlexNet · CIFAR10 batch 16 | test accuracy: 10%(explode) -> 69.05% | Improved |
| add spectral-criterion early stopping (FC2) | LeNet · MNIST batch 16 | test accuracy: 99.17% -> 99.08% | No clear effect |
| raise threshold constant from C=0.4 to C=0.6 (ablation) | spectral criterion · NN2 D1 synthetic | stopping epoch found: epoch 6 at C=0.4 -> - at C=0.6 (TP=0.02, FC1) | Mixed |
| decrease signal-to-noise ratio of Gaussian data (ablation) | NN1/NN2 · Gaussian K-class data | terminal spectrum type: Strong SNR: Light Tail (MP Law); Weak SNR: Heavy Tail | Hurt |
| increase number of classes K (ablation) | NN1/NN2 · Gaussian K-class data | heavy tail emergence: K=2 -> K=8: HT more likely at same SNR | Hurt |
| train on CIFAR10 instead of MNIST (ablation) | LeNet · batch 16 | spectrum type (detection rate): LT (99%) -> HT (64%) | Hurt |
| increase batch size (ablation) | VGG11 · CIFAR10 | spectrum type: spectra transit from BT to LT with increment of batch size | Improved |
| go to narrower deeper layers (FC2 to FC4) (ablation) | NN1 · Gaussian data | tail weight: FC2 -> FC4: tails of spectrum bulks become lighter | Improved |
Closest papers by meaning
- Traditional and Heavy Tailed Self Regularization in Neural Network Models (ICML 2019 · similarity 0.90)
- Models of Heavy-Tailed Mechanistic Universality (ICML 2025 · similarity 0.88)
- Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias (ICML 2025 · similarity 0.87)
- Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra (ICLR 2025 · similarity 0.86)
- Eigen Analysis of Conjugate Kernel and Neural Tangent Kernel (ICML 2025 · similarity 0.85)
- Free Decompression with Algebraic Spectral Curves (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
