Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Continuous-depth Deep Gaussian Processes
Ying Li, Sanyou Wu, Zi Yang, Qiaochu Xu, YUHAO LIU, Zhidi Lin, Michael Minyi Zhang, Petar Djuric, alexandre thiery
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes continuous-depth deep Gaussian processes in which the latent representation evolves as a stochastic differential equation over a continuous depth variable, rather than a discrete stack of GP layers.
Problem
Standard discrete-depth deep Gaussian processes become harder to optimize as depth increases.
Approach
Treat depth as a latent continuous evolution: the latent representation follows a stochastic differential equation with a depth-indexed dynamic, replacing a long discrete stack of GP layers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add residual (identity) connections to plain DGP (ablation) | DGP · benchmark regression (6 datasets) | RMSE/NLL: NLL 1.442 -> 1.254 on heteroscedastic | Improved |
| increase discrete depth L from 1 to 10 in plain DGP (ablation) | DGP · Toy regression | RMSE, NLL, MAE | Hurt |
| increase discrete depth L in residual DGP (ablation) | Residual DGP · Toy regression | RMSE, NLL, MAE | Mixed |
| vary discretization steps J from 1 to 10 (ablation) | CDGP · Toy regression | RMSE, NLL, MAE | No clear effect |
| vary terminal depth T from 1 to 10 (ablation) | CDGP · Toy regression | RMSE, NLL, MAE | No clear effect |
| use GP prior over drift instead of neural drift (NSDE) | continuous-depth SDE · benchmark regression (6 datasets) | NLL: NLL 3.527 -> 1.285 (heteroscedastic, NSDE vs CDGP) | Improved |
| evolve base drift with a CNF (FlowDGP) instead of direct separable GP prior (CDGP) | CDGP · benchmark regression (6 datasets) | train time (s): 201.330 -> 27.490 on heavy_tailed | Improved |
| replace CDGP with FlowDGP | CDGP · heteroscedastic, nonstationary | NLL: 1.285 -> 1.248; 0.979 -> 0.411 | Improved |
| use FlowDGP instead of CDGP | CDGP · sarcos_small | NLL: 0.376 -> 1.352 | Hurt |
| use continuous-depth models instead of TGP | TGP · 6 benchmark datasets | RMSE | Improved |
| use FlowDGP instead of NSDE on large-scale uci_bike | NSDE · uci_bike (full) | RMSE, NLL, CRPS | Improved |
| use FlowDGP instead of NSDE on large-scale uci_energy | NSDE · uci_energy (full) | NLL | Hurt |
Experimental setup
Builds on: deep Gaussian processes neural ODE
Closest papers by meaning
- Sparse Flows: Pruning Continuous-depth Models (NeurIPS 2021 · similarity 0.88)
- Efficient Continuous-Depth Modeling with GRU Equivalents (ICML 2026 · similarity 0.88)
- Deep Gaussian Processes for Regression using Approximate Expectation Propagation (ICML 2016 · similarity 0.87)
- Residual Deep Gaussian Processes on Manifolds (ICLR 2025 · similarity 0.85)
- Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference (ICML 2024 · similarity 0.85)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
