Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Residual Deep Gaussian Processes on Manifolds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes residual deep Gaussian processes on Riemannian manifolds, built from manifold-to-manifold layers defined as exponential maps of Gaussian vector fields, and shows gains on synthetic sphere regression, Bayesian optimisation, and wind interpolation.
Problem
Shallow Gaussian processes on manifolds struggle to model complex, irregular functions and vector fields, and no practical expressive deep GP framework exists for manifold-valued hidden layers.
Approach
Each hidden layer is a residual map f(x)=exp_x(g(x)) where g is a sparse Gaussian vector field, generalising the Euclidean residual deep GP of Salimbeni and Deisenroth to manifolds; the last layer can be scalar, manifold, or vector-valued, and inference uses doubly stochastic variational inference with interdomain inducing variables on compact manifolds.
Key results
- On wind interpolation, residual deep GPs significantly lower NLPD and MSE versus the shallow Hodge GVF baseline across three altitudes.
- On the synthetic S2 benchmark, residual deep GPs with Hodge GVFs and interdomain variables give the best NLPD; residual deep GPs are never worse than shallow counterparts.
- In Bayesian optimisation of the irregular S2 function, switching to a residual deep GP after 180 shallow iterations substantially reduces log regret, while the S3 Ackley results show no difference.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Hodge GVFs with interdomain inducing variables instead of other tested synthetic-model variants (ablation) | Residual deep GP · Synthetic sphere regression | NLPD | Helped |
| Use interdomain variables rather than inducing locations (ablation) | Residual deep GP · Synthetic sphere regression | NLPD | Helped |
| Use projected GVFs with interdomain variables (ablation) | Residual deep GP · Synthetic sphere regression | NLPD | Helped |
| Add residual manifold-to-manifold layers instead of using the shallow geometry-aware GP | Residual deep GP · Synthetic sphere regression with abundant data | NLPD | Helped |
| Add residual manifold-to-manifold layers instead of using the shallow geometry-aware GP | Residual deep GP · Synthetic sphere regression with sparse data | NLPD | No clear effect |
| Switch from a shallow geometry-aware GP to a residual deep GP in later optimization iterations | Bayesian optimization · Irregular sphere function | log regret | Helped |
| Switch from a shallow geometry-aware GP to a residual deep GP in later optimization iterations | Bayesian optimization · S^3 Ackley | median regret | No clear effect |
| Use the residual deep GP from the first optimization iteration rather than starting with a shallow GP (ablation) | Bayesian optimization · Irregular sphere function | log regret | Mixed |
| Add residual deep layers to the shallow Hodge GVF | Wind velocity model · Wind interpolation on the globe | NLPD | Helped |
| Map Euclidean inputs to a proxy sphere and use residual deep GPs with interdomain variables rather than Euclidean deep GPs with inducing points | Deep GP inference · UCI regression datasets | training-step wall-clock time | Helped |
| Map Euclidean inputs to a proxy sphere and use residual deep GPs rather than Euclidean deep GPs | Deep GP regression · UCI regression datasets | NLPD | Hurt |
Surprising findings
- On UCI data mapped to hyperspheres, Euclidean deep GPs always outperform residual deep GPs with the same number of layers in NLPD and MSE, though residual deep GPs train faster per step.
- Increasing the number of eigenfunctions K beyond the number of interdomain inducing variables degrades residual deep GP performance.
Limitations
- Closed-form eigenfields for Hodge GVFs are only available for the circle, the 2-sphere, and products of these, so other manifolds need other GVF constructions.
- Euclidean deep GPs outperform residual deep GPs on UCI regression after mapping inputs to a proxy hypersphere.
- Bayesian optimisation with pure deep GP is sensitive to setup and can get stuck in local minima.
Experimental setup
Datasets: ERA5 Aeolus satellite track UCI regression datasets synthetic spherical benchmark
Benchmarks: synthetic S2 regression S2 irregular Bayesian optimisation S3 Ackley Bayesian optimisation ERA5 wind interpolation UCI regression
Metrics: NLPD MSE log regret
Baselines: shallow Hodge GVF shallow geometry-aware GP Euclidean deep GP with inducing locations projected GVF residual deep GP
Builds on: Salimbeni and Deisenroth deep GP Hodge Matérn Gaussian vector fields doubly stochastic variational inference geometry-aware Matérn GPs (Borovitskiy et al.)
Closest papers by meaning
- Implicit Gaussian process representation of vector fields over arbitrary latent manifolds (ICLR 2024 · similarity 0.90)
- Vector-valued Gaussian Processes on Riemannian Manifolds via Gauge Independent Projected Kernels (NeurIPS 2021 · similarity 0.90)
- Implicit Manifold Gaussian Process Regression (NeurIPS 2023 · similarity 0.88)
- Learning Distributions on Manifolds with Free-Form Flows (NeurIPS 2024 · similarity 0.87)
- Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes (ICML 2024 · similarity 0.87)
- Building Transformation Layers for Riemannian Neural Networks (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.