Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Solving High Frequency and Multi-Scale PDEs with Gaussian Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GP-HM, a grid-based Gaussian process PDE solver whose spectral-mixture (student-t or Gaussian mixture) kernel is learned with log-weight Jeffreys-prior sparsity, using Kronecker structure to scale to massive collocation grids.
Problem
PINNs struggle with high-frequency and multi-scale PDE solutions due to spectral bias, and random Fourier feature fixes are sensitive to hyperparameters; existing GP PDE solvers use SE/Matérn kernels that fail to capture these frequencies.
Approach
Model the PDE solution's power spectrum as a student-t or Gaussian mixture, derive the covariance via the Wiener-Khinchin theorem (spectral mixture / spectral Matérn-mixture kernel), optimize log-weights (equivalent to a Jeffreys prior) to prune frequencies, and place collocation points on a grid with product kernels so the covariance factorizes into a Kronecker product, avoiding low-rank approxim
Key results
- GP-HM achieves relative L2 errors around 1e-3 to 1e-7 across most Poisson, Allen-Cahn and advection cases, while PINN and W-PINN basically fail (errors ~1e0).
- On 1D Poisson u2, RFF-PINN (2.00e-5) beats GP-HM (2.71e-3); Chebfun is more accurate on 1D Poisson/Allen-Cahn but is limited to 1D and temporal problems.
- Learned weights prune most of the Q=30 components (weights below 1e-10) and recovered frequencies closely match ground truth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the learned spectral-mixture covariance (GM or StM) instead of a plain SE or Matérn kernel (ablation) | GP-HM · High-frequency and multi-scale PDE benchmarks | relative L2 error: GP solvers using the plain SE and Matérn kernel result in several orders of the magnitude bigger errors | Helped |
| Use the Student-t mixture covariance instead of the Gaussian mixture covariance (ablation) | GP-HM · PDE benchmark suite | solution accuracy | Mixed |
| Increase the grid resolution to a sufficiently high level (ablation) | GP-HM · 1D and 2D Poisson with high-frequency solutions | solution accuracy | Helped |
| Use random Fourier features with PINN instead of a standard PINN | PINN · High-frequency and multi-scale PDE benchmarks | relative L2 error | Helped |
| Use GP-HM instead of competing machine-learning solvers | PDE solvers · High-frequency and multi-scale PDE benchmarks | relative L2 error | Mixed |
Surprising findings
- Second-order optimizers such as L-BFGS perform badly on the component selection task, while first-order ADAM works well.
- GP-HM takes tens of seconds to thousands of seconds of wall-clock time, slower than traditional numerical solvers such as Chebfun and finite difference.
Limitations
- Convergence needs many iterations (tens of thousands) for component selection and frequency adjustment.
- Relies on first-order optimization; second-order L-BFGS performs poorly, possibly due to local optima.
- Limited to grid-structured collocation points and product kernels; ML solvers are slower than traditional numerical methods.
Experimental setup
Benchmarks: 1D Poisson 2D Poisson 1D Allen-Cahn 2D Allen-Cahn 1D advection
Metrics: relative L2 error running time
Baselines: PINN W-PINN RFF-PINN Rowdy Spectral method Chebfun Finite Difference GP-SE GP-Matérn
Builds on: physics-informed neural networks spectral mixture kernel Wiener-Khinchin theorem random Fourier features PINN
Closest papers by meaning
- Gaussian Process Priors for Systems of Linear Partial Differential Equations with Constant Coefficients (ICML 2023 · similarity 0.88)
- gp2Scale: A Class of Compactly Supported Non-Stationary Kernels and Distributed Computing for Exact Gaussian Processes on 10 Million Data Points (ICML 2026 · similarity 0.88)
- A Generalized Stochastic Variational Bayesian Hyperparameter Learning Framework for Sparse Spectrum Gaussian Process Regression (AAAI 2017 · similarity 0.87)
- Solving High-Dimensional PDEs with Latent Spectral Models (ICML 2023 · similarity 0.87)
- Bias-Spectrum Neural Processes for Parametric PDEs: Architecture Priors Meet PDE Constraints (ICML 2026 · similarity 0.87)
- Orthogonally Decoupled Variational Gaussian Processes (NeurIPS 2018 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.