Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Neural operators meet conjugate gradients: The FCG-NO method for efficient PDE solving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a discretization-invariant spectral neural operator as a nonlinear preconditioner for flexible conjugate gradient (FCG) on 2D elliptic PDEs, using an energy-norm (Notay) loss and residuals sampled from Krylov subspaces.
Problem
Deep learning PDE solvers have limited accuracy (about 0.1%-1% relative L2 error), and classical preconditioners for large sparse SPD linear systems from elliptic PDEs are restrictive in form and slow to converge.
Approach
Use a spectral neural operator as a nonlinear preconditioner inside FCG, whose convergence theory tolerates nonlinear preconditioners close enough to a linear one; train it with an energy-norm loss (Notay loss) on residuals drawn from CG Krylov subspaces so it transfers across resolutions.
Key results
- On Poisson grid 128, NO+FCG reaches residual reduction 1e-6 in 20 iterations versus 216 for CG and 37 for GS(4) (best classical).
- Notay-loss-trained NO needs 20 iterations to reach 1e-6 at grid 128 versus 86 for L2-loss-trained NO on Poisson.
- NO trained on 32 or 64 grids and tested on 128 or 256 grids still converges, e.g., 30 iterations to 1e-3 for train 32 / test 128 on Poisson.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NO as a nonlinear FCG preconditioner instead of classical CG | FCG-NO · Poisson and Diffusion, grids 32–128 | iterations to residual 10^-6: 74 -> 9; 130 -> 14; 216 -> 20; 75 -> 9; 132 -> 14; 215 -> 19 | Helped |
| Use NO+FCG instead of the strongest listed classical preconditioner, GS(4) | FCG-NO · Poisson and Diffusion, grids 32–128 | iterations to residual 10^-6: 13 -> 9; 23 -> 14; 37 -> 20; 13 -> 9; 22 -> 14; 36 -> 19 | Helped |
| Replace Notay loss with L2 loss (ablation) | NO+FCG · Poisson, grid 32; residual target 10^-6 | iterations: 9 -> 15 | Hurt |
| Replace Notay loss with L2 loss (ablation) | NO+FCG · Poisson, grid 128; residual target 10^-6 | iterations: 20 -> 86 | Hurt |
| Replace Notay loss with L2 loss (ablation) | NO+FCG · Diffusion, grid 128; residual target 10^-6 | iterations | Hurt |
| Replace Krylov-subspace residuals with random right-hand-side residuals (ablation) | NO+FCG · Poisson, grid 32; residual target 10^-6 | iterations: 9 -> 10 | No clear effect |
| Replace Krylov-subspace residuals with random right-hand-side residuals (ablation) | NO+FCG · Poisson, grids 64 and 128; residual target 10^-6 | iterations: 14 -> 18; 20 -> 49 | Hurt |
| Replace Krylov-subspace residuals with random right-hand-side residuals (ablation) | NO+FCG · Diffusion, grids 64 and 128; convergence | iterations: --- | Hurt |
| Train NO at lower resolution and apply it at higher resolution | FCG-NO · Poisson and Diffusion, test grids up to 256 | iterations to residual reduction: 9; 18; 39 | Helped |
Surprising findings
- Residuals drawn from random right-hand sides instead of Krylov subspaces severely degrade training, with no convergence for Diffusion at grid 64 and above.
Limitations
- Requires the matrix A to be symmetric positive definite because CG is used
- Scalability to higher resolutions is limited by GPU memory
- Only uniform meshes and 2D problems are studied
Experimental setup
Datasets: synthetic Poisson elliptic problems (random trigonometric polynomials) synthetic Diffusion elliptic problems (random trigonometric polynomial coefficients)
Benchmarks: Poisson 2D elliptic (grids 32, 64, 128) Diffusion 2D elliptic (grids 32, 64, 128)
Metrics: number of FCG iterations to reduce residual by 10^-3, 10^-6, Notay loss per iteration
Baselines: CG Jacobi Gauss-Seidel ILU L2-loss neural operator preconditioner U-Net
Builds on: Fourier neural operator flexible conjugate gradient Notay nonlinear preconditioner theory
Closest papers by meaning
- Learning Sparse Approximate Inverse Preconditioners for Conjugate Gradient Solvers on GPUs (NeurIPS 2025 · similarity 0.88)
- An operator preconditioning perspective on training in physics-informed machine learning (ICLR 2024 · similarity 0.87)
- Toward Efficient Kernel-Based Solvers for Nonlinear PDEs (ICML 2025 · similarity 0.87)
- Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning (NeurIPS 2024 · similarity 0.86)
- Active Learning for Neural PDE Solvers (ICLR 2025 · similarity 0.86)
- Learning a Neural Solver for Parametric PDEs to Enhance Physics-Informed Methods (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.