Research map / Optimization / Implicit bias and generalization
Shadowing Properties of Optimization Algorithms
Antonio Orvieto, Aurelien Lucchi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Allow the shadowing orbit to start from a slightly perturbed initial condition | GD and Heavy-ball ODE approximations · Strongly-convex objectives and hyperbolic regions | uniform trajectory closeness | Improved |
| Start Gradient Descent at the exact ODE initial condition instead of perturbing it | Gradient Descent near an unstable saddle direction · Hyperbolic saddle | trajectory deviation | Hurt |
| Increase regularization | Gradient Descent on MNIST digits 3 and 5 · MNIST digits 3 and 5, regularized sigmoid loss | trajectory approximation error | Mixed |
| Use Heavy-ball momentum rather than Gradient Descent | MNIST digits 3 and 5 · MNIST digits 3 and 5, regularized sigmoid loss | convergence speed | Improved |
| Use Heavy-ball momentum rather than Gradient Descent | MNIST digits 3 and 5 · MNIST digits 3 and 5, regularized sigmoid loss | shadowing radius: 5 times bigger | Hurt |
| Use Heavy-ball instead of Gradient Descent | MNIST digits 3 and 5 · MNIST digits 3 and 5, regularized sigmoid loss | decay speed of approximation error | Improved |
| Use stochastic gradients with bounded gradient error | Gradient Descent ODE approximation · Strongly-convex objectives | shadowing: h≤2(εμ-R)/(ℓL) | Improved |
Closest papers by meaning
- The Physical Systems Behind Optimization Algorithms (NeurIPS 2018 · similarity 0.87)
- A General System of Differential Equations to Model First-Order Adaptive Algorithms (JMLR 2020 · similarity 0.87)
- Continuous-Time Analysis of Accelerated Gradient Methods via Conservation Laws in Dilated Coordinate Systems (ICML 2022 · similarity 0.86)
- Understanding the Curse of Unrolling (NeurIPS 2026 · similarity 0.85)
- Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions (ICML 2024 · similarity 0.85)
- Accelerating Optimization via Differentiable Stopping Time (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
