Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Accelerating Data Generation for Neural Operators via Krylov Subspace Recycling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SKR, which greedily sorts PDE-derived linear systems by parameter similarity and solves them sequentially with GCRO-DR Krylov subspace recycling to speed up neural operator training data generation.
Problem
Generating labeled PDE solution data for training neural operators requires solving many large sparse linear systems independently, which is very time-consuming.
Approach
Sort the parameterized linear systems so consecutive systems are highly similar (greedy nearest-neighbor on parameter matrices), then solve them sequentially with GCRO-DR Krylov subspace recycling, reusing approximate eigenvectors from the previous solve.
Key results
- SKR achieves up to 13.9x speedup in solve time over GMRES, per the abstract and introduction.
- Across Darcy, Thermal, Poisson and Helmholtz with 7 preconditioners and multiple tolerances, SKR reduces wall-clock time by roughly 2x to 14x and iterations by up to 30x (Helmholtz).
- Ablation on Darcy (SOR, size 1e4, tol 1e-8): sorting cuts time from 0.114s to 0.101s and iterations from 202.5 to 183.9.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SKR instead of GMRES | linear systems for Darcy Flow · Darcy Flow, 6400, 1e-8, None | time speedup: 2.70 | Helped |
| Use SKR instead of GMRES | linear systems for Thermal · Thermal, 11063, 1e-11, None | time speedup: 5.47 | Helped |
| Use SKR instead of GMRES | linear systems for Poisson · Poisson, 71313, 1e-5, ICC | time speedup: 0.99 | Hurt |
| Use SKR instead of GMRES | linear systems for Helmholtz · Helmholtz, 10000, 1e-7, ASM | time speedup: 13.9 | Helped |
| Add sorting to SKR (ablation) | SKR on Darcy Flow · Darcy Flow, 10000, SOR, 1e-8 | time (s): 0.114 -> 0.101 | Helped |
| Add sorting to SKR (ablation) | SKR on Darcy Flow · Darcy Flow, 10000, SOR, 1e-8 | iterations: 202.5 -> 183.9 | Helped |
| Add sorting to SKR (ablation) | SKR on Darcy Flow · Darcy Flow, 10000, SOR, 1e-8 | delta: 0.95 -> 0.90 | Helped |
Surprising findings
- Acceleration is less pronounced for low-accuracy Poisson problems with ICC or ILU preconditioning, since incomplete factorizations disrupt similarity between consecutive systems.
Limitations
- Focuses on linear PDEs and non-symmetric systems; other PDE types need tailored recycling methods.
- Sorting uses a simple greedy heuristic with Frobenius-norm distance on parameter matrices, not necessarily optimal.
- Acceleration diminishes for low-accuracy Poisson with ICC/ILU preconditioners.
Experimental setup
Datasets: Darcy flow linear systems (GRF permeability) Thermal steady-state FEM systems Poisson equation systems (Chebyshev polynomial boundary/source) Helmholtz equation systems (GRF wavenumber)
Benchmarks: Darcy flow Thermal problem Poisson equation Helmholtz equation
Metrics: computation time speed-up ratio iteration count speed-up ratio delta (one-sided distance)
Baselines: GMRES
Builds on: GCRO-DR GMRES Fourier Neural Operator
Closest papers by meaning
- Neural Krylov Iteration for Accelerating Linear System Solving (NeurIPS 2024 · similarity 0.88)
- Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning (NeurIPS 2024 · similarity 0.88)
- MgNO: Efficient Parameterization of Linear Operators via Multigrid (ICLR 2024 · similarity 0.87)
- Improved Operator Learning by Orthogonal Attention (ICML 2024 · similarity 0.86)
- Learning semilinear neural operators: A unified recursive framework for prediction and data assimilation. (ICLR 2024 · similarity 0.85)
- Active Learning for Neural PDE Solvers (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.