Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Physics-Informed Variational State-Space Gaussian Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PhysS-GP, a variational spatio-temporal state-space Gaussian process that incorporates linear and non-linear PDE/ODE constraints with linear-in-time Kalman smoothing, plus sparse, structured and spatial mini-batch approximations.
Problem
Physics-informed Gaussian process regression for spatio-temporal differential equations suffers from cubic computational cost in space-time and is limited to temporal settings or soft constraints; existing methods (AutoIP, Helmholtz-GP) scale poorly.
Approach
Combines Kronecker-structured spatio-temporal derivative GP priors written as a state-space LTI-SDE with conjugate variational inference via natural gradients, so the natural-gradient update reduces to Kalman smoothing that is linear in time; adds inducing points, structured posteriors, spatial mini-batching, and a Gauss-Newton PSD fix for non-linear PDEs.
Key results
- On the damped pendulum with 1000 collocation points, AutoIP is about 39 times slower than PhysS-GP (RMSE 0.06 vs 0.36 un-whitened AutoIP).
- On the diffusion-reaction system, PhysS-EKS achieves RMSE 0.09, NLPD -1.26, CRPS 0.038, in a single epoch, outperforming AutoIP (RMSE 0.17, 20000 epochs).
- On the Gulf of Mexico ocean currents with N=42,243, PhysS-SVGP_H reaches RMSE 0.14, NLPD -0.52, CRPS 0.078, at 1.86 s per epoch; Helmholtz-GP is infeasible at this scale.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase collocation points from 10 to 1000 for physs-gp (ablation) | physs-gp on the damped pendulum · Damped pendulum test | RMSE: 0.22 -> 0.06 | Helped |
| Increase collocation points from 10 to 1000 for autoip (ablation) | autoip on the damped pendulum · Damped pendulum test | Time (s): 153.52 -> 5656.62 | Hurt |
| Whiten the autoip approximation (ablation) | autoip on the damped pendulum, C=100 · Damped pendulum test | RMSE: 0.2 -> 0.05 | Helped |
| Use physs-gp instead of Helmholtz-GP | Curl-free magnetic field · Curl-free magnetic field | R Squared: 0.23 -> 0.21; 0.97 -> 0.97; 0.97 -> 0.97 | No clear effect |
| Add spatial inducing-point sparsity (ablation) | physs-gp on the curl-free magnetic field · Curl-free magnetic field, spatial size 20 | R Squared: 0.97 -> 0.97 | No clear effect |
| Use the structured variational posterior (ablation) | physs-gp on the curl-free magnetic field · Curl-free magnetic field, spatial size 20 | R Squared: 0.97 -> 0.74 | Hurt |
| Use the EKS prior (ablation) | physs-GP on the diffusion-reaction system · Diffusion-reaction system | RMSE: 0.19 -> 0.09 | Helped |
| Use spatial inducing-point sparsity (ablation) | physs-GP on the diffusion-reaction system · Diffusion-reaction system | RMSE: 0.09 -> 0.19 | Hurt |
| Use the structured variational posterior (ablation) | physs-SVGP on the diffusion-reaction system · Diffusion-reaction system | RMSE: 0.19 -> 0.17 | Helped |
Surprising findings
- Un-whitened AutoIP performs poorly on the pendulum, suggesting optimisation difficulty from non-linearity rather than model limitations.
- The structured approximation on the magnetic field experiment drops to R squared 0.61 at spatial size 10 for PhysS-SVGP_H, compared with 0.97 for exact PhysS-GP.
Limitations
- Collocation method only enforces the differential equation pointwise.
- State size remains the computational bottleneck.
- Approximations (sparse, structured) can reduce predictive performance, e.g. on the magnetic field experiment.
Experimental setup
Datasets: simulated damped pendulum simulated magnetic dipole field Allen-Cahn diffusion-reaction system Gulf of Mexico buoy ocean currents
Metrics: RMSE NLPD CRPS wall-clock time R squared
Baselines: AutoIP Helmholtz-GP batch GP
Builds on: AutoIP Helmholtz-GP probabilistic meshless method conjugate variational inference extended Kalman smoother
Closest papers by meaning
- Free-Form Variational Inference for Gaussian Process State-Space Models (ICML 2023 · similarity 0.90)
- Spatio-Temporal Variational Gaussian Processes (NeurIPS 2021 · similarity 0.89)
- AutoIP: A United Framework to Integrate Physics into Gaussian Processes (ICML 2022 · similarity 0.89)
- Sparse Gaussian Processes: Structured Approximations and Power-EP Revisited (NeurIPS 2025 · similarity 0.86)
- Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference (NeurIPS 2024 · similarity 0.86)
- Exact Inference for Continuous-Time Gaussian Process Dynamics (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.