Research map / Optimization / Optimizers
A Differentiable Interior-Point Method in Single Precision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an implicit-complementarity primal-dual interior-point QP solver and differentiation scheme that stays spectrally bounded, enabling reliable solving and differentiation of QPs in single precision.
Problem
Standard primal-dual interior-point methods produce increasingly ill-conditioned linear systems near the solution, which become catastrophic in single precision, blocking differentiable IPM QP solvers from running on low-precision ML hardware.
Approach
Replace explicit complementarity z*s=kappa with an implicit representation using a softplus retraction map b_kappa(v) with b(v)b(-v)=kappa, yielding a partially condensed KKT system whose only varying block is a diagonal matrix bounded in (0,1), so eigenvalues stay bounded; differentiate at a relaxed central-path point reusing factorizations.
Key results
- On polytope projection with n>5, explicit f32 returned NaN gradients in 50% of computations while implicit f32 had 0% failure and gradient error near the kappa_relax floor.
- In bilevel trajectory optimization, implicit f32 matched f64 performance while explicit f32 produced unusable gradients or failed at kappa_relax=1e-4.
- Implicit f32 allowed GPU batch sizes twice as large as implicit f64 in the safety-filter learning experiment, while explicit f32 produced NaNs for more than one agent.
Surprising findings
- Explicit f32 failed for any safety-filter scenario with more than one agent, producing NaNs.
Limitations
- Restricted to quadratic programs; extension to other convex cones left to future work.
- Densifies the Hessian in the partially condensed form (Q - G^T G).
- Relies on a relaxation parameter kappa_relax whose choice trades off smoothing and accuracy.
Experimental setup
Datasets: synthetic polytope projection instances forest and office bilevel trajectory environments multi-agent safety filter simulation instances
Benchmarks: synthetic polytope projection gradient error bilevel trajectory optimization navigation time (forest, office) multi-agent safety filter training loss
Metrics: gradient error NaN rate navigation time training loss maximum batch size
Baselines: explicit primal-dual IPM (f32) explicit primal-dual IPM (f64) implicit f64 vanilla MLP
Builds on: CVXPY Layers OptNet qpax
Closest papers by meaning
- Solving Constrained Variational Inequalities via a First-order Interior Point-based Method (ICLR 2023 · similarity 0.88)
- Barrier Algorithms for Constrained Non-Convex Optimization (ICML 2024 · similarity 0.86)
- Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity (NeurIPS 2025 · similarity 0.85)
- Directional Smoothness and Gradient Methods: Convergence and Adaptivity (NeurIPS 2024 · similarity 0.85)
- An Accelerated Gradient Method for Convex Smooth Simple Bilevel Optimization (NeurIPS 2024 · similarity 0.85)
- Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.