Research map / Optimization / Implicit bias and generalization
Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CIFAR-100 rather than CIFAR-10 | small-allcnn · CIFAR-10 and CIFAR-100 | diffusion eigenvalue mean ± standard deviation: 0.27 \pm 0.84 -> 0.98 \pm 2.16 | Mixed |
| Add data augmentation | small-allcnn · CIFAR-10 | diffusion eigenvalue mean ± standard deviation: 0.27 \pm 0.84 -> 0.43 \pm 1.32 | Helped |
| Add data augmentation | small-allcnn · CIFAR-10 | diffusion matrix rank: 0.34% -> 0.32% | No clear effect |
| Use a convolutional network instead of a fully connected network | MNIST · MNIST | diffusion matrix rank: 0.6% -> 1.8% | Helped |
| Use isotropic diffusion instead of non-isotropic diffusion | SGD dynamics · General | difference between implicit potential and original loss | Helped |
| Use non-isotropic mini-batch gradient noise | SGD dynamics for deep networks · Deep networks | trajectory periodicity | Helped |
| Increase learning rate relative to batch size | SGD · General | strength of implicit entropic regularization | Helped |
Closest papers by meaning
- SGD with Large Step Sizes Learns Sparse Features (ICML 2023 · similarity 0.91)
- Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks (NeurIPS 2023 · similarity 0.91)
- SGD Converges to Global Minimum in Deep Learning via Star-convex Path (ICLR 2019 · similarity 0.91)
- SGD Can Converge to Local Maxima (ICLR 2022 · similarity 0.91)
- A Variational Analysis of Stochastic Gradient Algorithms (ICML 2016 · similarity 0.91)
- The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.