Research map / Architectures and efficiency / RNNs and normalization
DiLQR: Differentiable Iterative Linear Quadratic Regulator via Implicit Differentiation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiLQR, which differentiates through the fixed point of iLQR via implicit differentiation, giving analytical gradients with constant backward cost in iteration count, and evaluates on pendulum/cartpole imitation and system identification.
Problem
Differentiating through iLQR by unrolling its iterations is memory- and time-costly and scales poorly with iterations and horizon, and prior differentiable-LQR methods (e.g., DiffMPC) treat the last-layer input as constant, yielding inexact gradients.
Approach
Treats the iLQR output as a fixed point (X*, U*) = (F(X*,U*,θ), G(X*,U*,θ)) and applies implicit differentiation to get closed-form Jacobians dX*/dθ and dU*/dθ, so backward cost is independent of iteration count; the method also accounts for the dependence of the linearization on the trajectory, unlike DiffMPC.
Key results
- Backward pass is up to 128x faster than AutoDiff (8.57 s vs 0.067 s at horizon 10, 300 iterations) and at least 21x faster (horizon 10, 50 iterations).
- Imitation loss in dx mode is about 10^6 times lower than the LSTM NN policy on pendulum/cartpole with train=100.
- Dynamics-learning model loss is 41% lower than DiffMPC with train=50; bad-value ratio 2.76% vs DiffMPC's 16.85% with train=100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use implicit differentiation instead of unrolled AutoDiff | iLQR backward pass · Pendulum, horizon 10, 300 iterations | backward computation time: 8.57 seconds -> 0.067 seconds; 128x speedup | Helped |
| Use implicit differentiation instead of unrolled AutoDiff | iLQR backward pass · Pendulum, horizon 10, 50 iterations | backward computation time: 1.41 seconds -> 0.067 seconds; 21x speedup | Helped |
| Learn dynamics with DiLQR.dx instead of an LSTM neural network policy | Imitation learning · Pendulum, train=100 | imitation loss: 10^6x | Helped |
| Learn dynamics with DiLQR.dx instead of an LSTM neural network policy | Imitation learning · CartPole, train=100 | imitation loss: 10^4 | Helped |
| Use DiLQR.cost instead of DiffMPC.cost | CartPole cost learning · CartPole | model loss: 32% | Helped |
| Use DiLQR.dx instead of DiffMPC.dx | CartPole dynamics learning · CartPole, train=50 | final model loss: 41% lower final error | Helped |
| Use DiLQR.dx instead of DiffMPC.dx | CartPole dynamics learning · CartPole, train=100 | final model loss | Mixed |
| Use DiLQR.dx instead of DiffMPC.dx | CartPole dynamics learning · CartPole, train=100 | bad-value ratio: 16.85% -> 2.76% | Helped |
| Remove the forward algorithm (ablation) | DiLQR · Pendulum, varying horizons | computation time | Hurt |
| Disable parallelization (ablation) | DiLQR · Pendulum, varying horizons | computation time | Hurt |
| Disable parallelization (ablation) | DiLQR · Pendulum, horizon 30 | computation time | Mixed |
| Remove sparsity exploitation (ablation) | DiLQR · Pendulum, varying horizons | computation time | No clear effect |
Surprising findings
- The authors disclose that an earlier rebuttal reported a 0.0% bad-value ratio that was generated with an AI-assisted summarization tool and was corrected to 2.76% in the final version.
Limitations
- Assumes iLQR converges to a fixed point.
- Requires access to first- and second-order derivatives of the dynamics.
- Experiments are on simple toy control tasks (pendulum, cartpole).
Experimental setup
Datasets: iLQR expert trajectories on pendulum and cartpole
Benchmarks: Pendulum CartPole
Metrics: backward computation time imitation test loss model loss MSE bad-value ratio image prediction error
Baselines: Neural network policy (LSTM) SysId DiffMPC SafePDP IDOC AutoDiff unrolled iLQR
Builds on: iLQR DiffMPC implicit differentiation SafePDP IDOC
Closest papers by meaning
- Safe-PG-LQR: Provably Safe, Convergent and Optimal Model-Free Linear Quadratic Regulator with Hard Constraints (NeurIPS 2026 · similarity 0.86)
- Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States (ICML 2024 · similarity 0.86)
- Revisiting Implicit Differentiation for Learning Problems in Optimal Control (NeurIPS 2023 · similarity 0.85)
- Differentiation Through Black-Box Quadratic Programming Solvers (NeurIPS 2025 · similarity 0.85)
- Differentiable Nonlinear Model Predictive Control (NeurIPS 2026 · similarity 0.83)
- Handling Heterogeneous Curvatures in Bandit LQR Control (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.