Research map / Optimization / Optimizers
One-step differentiation of iterative algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace unrolled automatic differentiation with one-step differentiation (stop_gradient on all but last iteration) | Newton's method for regularized logistic regression · Logistic regression hypergradient timing | computation time vs problem size | Helped |
| Use one-step differentiation instead of implicit differentiation | Newton's method for logistic regression · Logistic regression hypergradient timing | computation time | Helped |
| Use one-step instead of implicit differentiation for Jacobian accuracy | Newton's method for logistic regression · Logistic regression | Jacobian agreement with implicit differentiation: up to $10^{-12}$ error | No clear effect |
| Replace unrolled automatic differentiation with one-step differentiation | primal-dual interior point QP solver · Differentiable quadratic programs | computation time vs problem size | Helped |
| Use one-step instead of implicit differentiation for Jacobian accuracy | interior point solver for QP · Random QP instances | Jacobian agreement with implicit differentiation: up to order $10^{-6}$ | No clear effect |
| Use one-step differentiation instead of full unrolling | gradient descent on weighted ridge regression · cpusmall (LibSVM) | speed and Jacobian precision | Mixed |
| Use implicit differentiation instead of one-step differentiation | gradient descent on weighted ridge regression · cpusmall (LibSVM) | speed and Jacobian precision | Helped |
| Differentiate through the last K=1/kappa steps (F^K) instead of a single step for linearly convergent algorithms | gradient descent on ridge regression · cpusmall, condition numbers 1000 and 100 | Jacobian error | Helped |
| Use step size 2/(mu+L) instead of 1/L | gradient descent ridge regression · cpusmall, condition numbers 1000 and 100 | Jacobian and iterate optimality | Mixed |
| Unrolled automatic differentiation of iterations versus one-step | automatic differentiation · weighted ridge regression | Jacobian error over iterations | Hurt |
Closest papers by meaning
- Automatic Differentiation of Optimization Algorithms with Time-Varying Updates (ICML 2025 · similarity 0.88)
- Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates (ICML 2024 · similarity 0.86)
- Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization (ICML 2024 · similarity 0.86)
- Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity (NeurIPS 2025 · similarity 0.85)
- Accelerating Optimization via Differentiable Stopping Time (NeurIPS 2025 · similarity 0.85)
- Directional Smoothness and Gradient Methods: Convergence and Adaptivity (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.