Research map / Optimization / Implicit bias and generalization
Entropic gradient descent algorithms and wide flat minima
Fabrizio Pittorino, Carlo Lucibello, Christoph Feinauer, Gabriele Perugini, Carlo Baldassi, Elizaveta Demyanenko, Riccardo Zecchina
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use eSGD instead of SGD | SmallConvNet · CIFAR-10 | test error (%): 16.5 \pm 0.2 -> 14.7 \pm 0.3 | Improved |
| Use rSGD instead of SGD | ResNet-18 · CIFAR-10 | test error (%): 13.1 \pm 0.3 -> 12.4 \pm 0.3 | Improved |
| Use eSGD instead of SGD | ResNet-18 · CIFAR-10 | test error (%): 13.1 \pm 0.3 -> 12.1 \pm 0.3 | Improved |
| Use rSGD instead of SGD | ResNet-110 · CIFAR-10 | test error (%): 6.4 \pm 0.1 -> 6.2 \pm 0.2 | Improved |
| Use eSGD instead of SGD | ResNet-110 · CIFAR-10 | test error (%): 6.4 \pm 0.1 -> 6.2 \pm 0.1 | Improved |
| Use rSGD instead of SGD | PyramidNet+ShakeDrop · CIFAR-10 | test error (%): 2.1\pm0.2 -> 2.2\pm0.1 | Hurt |
| Use rSGD instead of SGD | PyramidNet+ShakeDrop · CIFAR-100 | test error (%): 13.8\pm0.1 -> 13.5\pm0.1 | Improved |
| Use rSGD instead of SGD | EfficientNet-B0 · CIFAR-100 | test error (%): 20.5 -> 20.6 | Hurt |
| Use eSGD instead of SGD | EfficientNet-B0 · CIFAR-100 | test error (%): 20.5 -> 20.1 \pm 0.2 | Improved |
| Use rSGD instead of SGD | ResNet-50 · Tiny ImageNet | test error (%): 45.2 \pm 1.2 -> 41.5 \pm 0.3 | Improved |
| Use eSGD instead of SGD | ResNet-50 · Tiny ImageNet | test error (%): 45.2 \pm 1.2 -> 41.7\pm1 | Improved |
| Use rSGD instead of SGD | DenseNet-121 · Tiny ImageNet | test error (%): 41.4 \pm 0.3 -> 39.8 \pm 0.2 | Improved |
Closest papers by meaning
- Entropy-SGD: Biasing Gradient Descent Into Wide Valleys (ICLR 2017 · similarity 0.91)
- Flat Minima and Generalization: Insights from Stochastic Convex Optimization (ICML 2026 · similarity 0.90)
- Sharp Minima Can Generalize For Deep Nets (ICML 2017 · similarity 0.89)
- A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias (NeurIPS 2025 · similarity 0.88)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.88)
- Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
