Research map / Optimization / Optimizers
The Curse of Unrolling: Rate of Differentiating Through Optimization
Damien Scieur, Gauthier Gidel, Quentin Bertrand, Fabian Pedregosa
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Sobolev-optimized residual polynomials instead of gradient descent or Chebyshev unrolling | Unrolled differentiation · Breast Cancer, bodyfat, and synthetic ridge regression | Jacobian suboptimality | Improved |
| Use gradient descent with h=2/(L+ℓ) rather than h=1/L | Gradient descent unrolling · Quadratic objectives with bounded spectrum | asymptotic Jacobian convergence rate: ((1−κ)/(1+κ))^(t−1) vs. (1−κ)^(t−1) | Improved |
| Use gradient descent with h=2/(L+ℓ) rather than h=1/L | Gradient descent unrolling · Quadratic objectives with bounded spectrum | burn-in and peak Jacobian suboptimality: Oκ→0(1/κ||∂x₀(θ)−∂x⋆(θ)||F) | Hurt |
| Use Chebyshev unrolling rather than gradient descent | Unrolled differentiation · Quadratic objectives with bounded spectrum | burn-in duration: t≈2√(1/κ) vs. t≈1/κ | Mixed |
| Apply the two-phase analysis to logistic regression rather than a quadratic objective | Unrolled differentiation · Synthetic and breast-cancer logistic regression | presence of burn-in followed by convergence | No clear effect |
| Apply the two-phase analysis without the commutativity assumption | Unrolled differentiation · Generalized ridge regression with diagonal matrix D | presence of burn-in followed by convergence | No clear effect |
| Choose a large gradient-descent step size for faster asymptotic convergence | Gradient descent unrolling · Quadratic objectives with bounded spectrum | early Jacobian suboptimality: Oκ→0(1/κ||∂x₀(θ)−∂x⋆(θ)||F) | Hurt |
Closest papers by meaning
- Understanding the Curse of Unrolling (NeurIPS 2026 · similarity 0.94)
- Automatic Differentiation of Optimization Algorithms with Time-Varying Updates (ICML 2025 · similarity 0.88)
- A Unified Convergence Analysis for Shuffling-Type Gradient Methods (JMLR 2021 · similarity 0.86)
- Shadowing Properties of Optimization Algorithms (NeurIPS 2019 · similarity 0.86)
- Accelerating Optimization via Differentiable Stopping Time (NeurIPS 2025 · similarity 0.85)
- Derivatives of Stochastic Gradient Descent in parametric optimization (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
