Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes non-intrusive gradient-based meta-solving (NI-GBMS), which trains neural meta-solvers to set parameters of non-differentiable legacy solvers (PETSc) using a control-variate forward gradient with an adaptively trained surrogate model.
Problem
Accelerating legacy, non-automatic-differentiable numerical solvers by learning solver parameters (e.g., initial guesses) with neural networks, where backpropagation through the solver is impossible.
Approach
Reduce the variance of the finite-difference forward gradient of a black-box solver using control variates built from the forward gradient of an online-trained neural surrogate of the solver, yielding an unbiased (asymptotically, for finite differences) gradient estimator without modifying the solver.
Key results
- On Poisson with Jacobi, the proposed method reduces iterations from 175.71 (supervised baseline) to 45.15, a 74% reduction, close to the true-gradient baseline (53.64).
- With multigrid on Poisson, iterations drop to 43.06 versus 78.62 for the supervised baseline (45%), while the true-gradient baseline reaches 39.29.
- On biharmonic and elasticity PETSc AMG problems, average iterations fall from 73.74 to 31.54 and from 88.71 to 25.61 respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the meta-solver with NI-GBMS instead of solver-independent supervised learning | Poisson equation with Jacobi · Poisson equation, Jacobi | iterations: 175.71 -> 45.15; 74% | Helped |
| Train the meta-solver with NI-GBMS instead of solver-independent supervised learning | Poisson equation with geometric multigrid · Poisson equation, geometric multigrid | iterations: 78.62 -> 43.06; 45% | Helped |
| Learn initial guesses with NI-GBMS rather than use the default zero guess | PETSc AMG · Biharmonic equation | iterations: 73.74 -> 31.54; 57% | Helped |
| Learn initial guesses with NI-GBMS rather than use the default zero guess | PETSc AMG · Linear elasticity equations | iterations: 88.71 -> 25.61; 71% | Helped |
| Use finite-difference control-variate forward gradients instead of exact control-variate forward gradients (ablation) | GBMS meta-solver training · Poisson equation, Jacobi and geometric multigrid | iterations: 44.92 -> 45.15; 42.76 -> 43.06 | No clear effect |
| Use control-variate forward gradients rather than ordinary forward gradients | GBMS meta-solver training · Poisson equation, Jacobi and geometric multigrid | iterations: 66.85 -> 45.15; 59.09 -> 43.06 | Helped |
Surprising findings
- The gradient of the surrogate model alone is unstable and in some cases (Rosenbrock d=8) outperforms the true gradient, though it is biased and generally worse than the proposed estimator.
Limitations
- Finite-difference version is only asymptotically unbiased and requires tiny step sizes, and the convergence analysis covers only strongly convex scalar objectives with exact forward gradients.
- Early training is slowed while the surrogate model fits the solver.
- Experiments use small problem sizes (e.g., N=31 for Poisson) and fully connected or convolutional surrogates without solver-informed architectures.
Experimental setup
Datasets: synthetic 1D Poisson tasks synthetic 2D biharmonic tasks synthetic 3D linear elasticity tasks
Benchmarks: Sphere function Rosenbrock function 1D Poisson equation with Jacobi and multigrid 2D biharmonic equation with AMG 3D linear elasticity with AMG
Metrics: objective value average number of iterations to converge cosine similarity to true gradient
Baselines: supervised learning initial guess true gradient training surrogate gradient training forward gradient with finite differences zero initial guess
Builds on: gradient-based meta-solving forward gradient PETSc Estimate and Replace
Closest papers by meaning
- Accelerating Optimization via Differentiable Stopping Time (NeurIPS 2025 · similarity 0.85)
- Scalable Meta-Learning via Mixed-Mode Differentiation (ICML 2025 · similarity 0.84)
- Metamizer: A Versatile Neural Optimizer for Fast and Accurate Physics Simulations (ICLR 2025 · similarity 0.84)
- Memory-Reduced Meta-Learning with Guaranteed Convergence (AAAI 2025 · similarity 0.83)
- $\bf{\Phi}_\textrm{Flow}$: Differentiable Simulations for PyTorch, TensorFlow and Jax (ICML 2024 · similarity 0.83)
- TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.