Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes graph neural preconditioners (GNPs), a scale-equivariant GCN trained to approximate A^{-1} from (b,x) pairs and used inside FGMRES as a general-purpose algebraic preconditioner, evaluated on over 800 SuiteSparse matrices.
Problem
General-purpose (algebraic) preconditioners for large sparse nonsymmetric linear systems are hard to design robustly; ILU and AMG often fail or have unpredictable construction cost on ill-conditioned matrices.
Approach
Train a residual graph convolutional network, with the sparse matrix as the graph adjacency, to map b to x = A^{-1}b, using Arnoldi-based sampling of training pairs and a parameter-free scale-equivariant input/output wrapper, and use it as a nonlinear preconditioner in flexible GMRES, with a convergence analysis for FGMRES with a nonlinear operator.
Key results
- GNP performs best on a substantial portion of the ~800 SuiteSparse matrices under Iter-AUC and Time-AUC, with ILU and AMG best on more problems overall.
- GNP had 0 construction failures and 1 solution failure out of 867 matrices, versus 348 construction failures for ILU (40.14%) and 62 for AMG (7.15%).
- On some ill-conditioned matrices GNP reduces the residual by many orders of magnitude relative to the second-best preconditioner; e.g., VanVelzen/std1_Jac3 converges in 4 iterations to 1e-8 where others stall above 1e-2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed mixture-based training data instead of sampling only x from N(0, I_n) or b from N(0, I_n) (ablation) | GNP · SuiteSparse matrices | relative residual norm | Helped |
| Sample x only from N(0, I_n) instead of the proposed mixture (ablation) | GNP · SuiteSparse matrices | training loss | Helped |
| Sample b from N(0, I_n) instead of the proposed mixture (ablation) | GNP · SuiteSparse matrices | training loss | Hurt |
| Add the scale-equivariant scaling operator s and back-scaling s^-1 (ablation) | GNP · SuiteSparse matrices | relative residual norm | Helped |
| Replace PyAMG's default SA solver with AIR for nonsymmetric matrices (ablation) | PyAMG · Nonsymmetric matrices | iteration count: 14 better, 8 worse, 61 equal | Mixed |
| Replace PyAMG's default SA solver with AIR for nonsymmetric matrices (ablation) | PyAMG · Nonsymmetric matrices | final residual: 50 better, 29 worse, 4 equal | Mixed |
| Use AmgX instead of PyAMG (ablation) | AMG · SuiteSparse 867-matrix evaluation | construction failures: 62 (7.15%) -> 105 (12.11%) | Hurt |
Surprising findings
- ILU fails on nearly half (40%) of the matrices at construction, and GNP is most useful on ill-conditioned matrices (condition number >= 1e16), where ILU is less competitive.
- Using x ~ N(0, I) for training yields lower loss but the Arnoldi-based mixture gives the best preconditioning performance.
Limitations
- Hyperparameters were fixed across all matrices without tuning, so performance may be understated.
- A single network is trained per matrix; it does not generalize across different matrices.
- GPU memory limits the matrix sizes and network complexity that can be tested (matrices restricted to 1K-100K rows and under 2M nonzeros).
Experimental setup
Datasets: SuiteSparse Matrix Collection (867 selected non-SPD real square matrices, 867 after selection, over 800 evaluated)
Benchmarks: SuiteSparse Matrix Collection
Metrics: Iter-AUC Time-AUC final relative residual norm construction time solution time failure count
Baselines: ILU AMG (PyAMG) GMRES as preconditioner (inner-outer GMRES) Jacobi AmgX
Builds on: GMRES flexible GMRES graph convolutional network ILU PyAMG
Closest papers by meaning
- Learning Sparse Approximate Inverse Preconditioners for Conjugate Gradient Solvers on GPUs (NeurIPS 2025 · similarity 0.90)
- SymMaP: Improving Computational Efficiency in Linear Solvers through Symbolic Preconditioning (NeurIPS 2025 · similarity 0.86)
- Expressive Power of Graph Neural Networks for (Mixed-Integer) Quadratic Programs (ICML 2025 · similarity 0.86)
- Graph Neural PDE Solvers with Conservation and Similarity-Equivariance (ICML 2024 · similarity 0.84)
- A Neural-Preconditioned Poisson Solver for Mixed Dirichlet and Neumann Boundary Conditions (ICML 2024 · similarity 0.84)
- Universal Sequence Preconditioning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.