Research map / Learning theory and algorithms / Combinatorial and search optimization
Neural Krylov Iteration for Accelerating Linear System Solving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NeurKItt, which uses a Fourier Neural Operator with QR decomposition and a projection loss to predict an invariant subspace that is used as a deflation space to accelerate GMRES-style Krylov iteration on PDE-derived sparse linear systems.
Problem
Krylov subspace solvers (e.g., GMRES) for large sparse non-symmetric linear systems require many iterations because iteration starts from a random vector; prior learned accelerators mostly target symmetric positive definite systems or initial-guess/preconditioner prediction.
Approach
Predict the invariant subspace associated with the smallest eigenvalues of A using a neural operator (FNO) with thin QR orthogonalization and a projection loss, then use this predicted subspace as a deflation space inside Krylov iteration (recycling-style), rather than predicting an initial solution.
Key results
- Up to 5.5x time speedup and 16.1x iteration speedup over GMRES across preconditioners and tolerances (Table 1, Darcy with no preconditioning at tol 1e-2).
- Minimum time speedup of 1.67 on Darcy flow across tested preconditioners, per the paper.
- Ablation: replacing projection loss with MSE or removing QR yields principal angle 1.57 rad versus 0.07 rad for full NeurKItt.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use NeurKItt instead of GMRES | linear system solver · Darcy Flow, tol 1e-2, no preconditioner | time speedup: 5.55 | Helped |
| use NeurKItt instead of GMRES | linear system solver · Darcy Flow, tol 1e-2, no preconditioner | iteration speedup: 16.13 | Helped |
| replace FNO with MLP (ablation) | NeurKItt subspace prediction · Darcy Flow, matrix size 32400 | principal angle (rad): 0.07 -> 0.54 | Hurt |
| remove QR decomposition (ablation) | NeurKItt subspace prediction · Darcy Flow, matrix size 32400 | principal angle (rad): 0.07 -> 1.57 | Hurt |
| replace projection loss with MSE loss (ablation) | NeurKItt subspace prediction · Darcy Flow, matrix size 32400 | principal angle (rad): 0.07 -> 1.57 | Hurt |
| use 2 instead of 5 FNO layers (ablation) | NeurKItt · Darcy Flow, matrix size 32400, tol 1e-5, no preconditioner | time speedup: 1.68 -> 1.47 | Hurt |
| use 8 instead of 5 FNO layers (ablation) | NeurKItt · Darcy Flow, matrix size 32400, tol 1e-5, no preconditioner | time speedup: 1.68 -> 1.69 | No clear effect |
| use 10 instead of 5 FNO layers (ablation) | NeurKItt · Darcy Flow, matrix size 32400, tol 1e-5, no preconditioner | time speedup: 1.68 -> 1.54 | Hurt |
| increase Fourier modes from 8 to 64 (ablation) | NeurKItt · Darcy Flow, matrix size 32400, tol 1e-5, no preconditioner | time speedup: 1.19 -> 1.71 | Helped |
| increase width from 8 to 64 (ablation) | NeurKItt · Darcy Flow, matrix size 32400, tol 1e-5, no preconditioner | time speedup: 0.98 -> 1.83 | Helped |
| increase subspace dimension (ablation) | NeurKItt · Darcy Flow, matrix size 32400 | iteration speedup | Helped |
Surprising findings
- Using MSE loss on subspace prediction was ineffective: predicted subspace became essentially orthogonal to the target.
- Time speedup peaks around subspace dimension 8 and decreases for larger dimensions even though iteration speedup keeps increasing.
Limitations
- Only handles generic non-symmetric matrices; structured matrices need tailored methods.
- Does not study how different preconditioners affect the predicted subspace.
- Evaluated only on three PDE-derived datasets with a fixed FNO setup and small subspace dimension (10).
Experimental setup
Datasets: Helmholtz equation linear systems (generated) Darcy flow linear systems (generated, GRF coefficients) Non-uniform heat conduction linear systems (generated)
Benchmarks: Helmholtz (62500) Darcy flow (160000) Non-uniform heat conduction (90000)
Metrics: time speedup iteration speedup principal angle
Baselines: GMRES (PETSc 3.19.4) FNO-MLP variant (ablation) MSE loss variant (ablation)
Builds on: Fourier Neural Operator GMRES Krylov subspace recycling PETSc
Closest papers by meaning
- Accelerating Data Generation for Neural Operators via Krylov Subspace Recycling (ICLR 2024 · similarity 0.88)
- Learning to Discover Iterative Spectral Algorithms (NeurIPS 2026 · similarity 0.84)
- SymMaP: Improving Computational Efficiency in Linear Solvers through Symbolic Preconditioning (NeurIPS 2025 · similarity 0.83)
- Schwarz–Schur Involution: Lightspeed Differentiable Sparse Linear Solvers (ICML 2025 · similarity 0.83)
- Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems (ICLR 2025 · similarity 0.83)
- Provable Acceleration of Nesterov's Accelerated Gradient for Asymmetric Matrix Factorization and Linear Neural Networks (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.