Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Scalable Derivative Gaussian Processes via Exact Gradient Reduction

NeurIPS 2026 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Sparse variational Gaussian processes · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TERA, a scalable derivative GP that reduces observed gradients to target-specific directional statistics inside Vecchia factors, giving exact conditionals for stationary kernels with cost independent of d.

Problem

Exact derivative GP inference with n function values and n full d-dimensional gradients scales cubically in the joint state size (O(n^3 d^3)), making it intractable for large n or d.

Approach

For stationary kernels, gradient components orthogonal to target-centered difference directions are conditionally irrelevant to a target function value, so each Vecchia factor only needs at most m^2 reduced directional-derivative statistics, leaving the derivative GP unchanged while removing dependence on d.

Key results

• On MD22 energy regression, TERA gives the lowest raw energy RMSE per atom across six molecular systems (up to 62,777 samples and 1,110 dimensions) while Exact GP and DSoftKI fail to complete on DHA.
• In GP simulations, TERA matches the posterior KL fidelity of full-gradient Vecchia derivative GPs while being much faster and staying near Exact GP timing across d and n sweeps.
• On Ackley-200D, the full BO run takes about five minutes with peak GPU memory below one GB, and TERA reaches lower final regret than Sobol, VBO and TuRBO-1 within 150 evaluations.

Surprising findings

• One pass over training Vecchia factors suffices for TERA to reach reported accuracy, whereas DDSVGP and DSoftKI need multiple epochs.

Limitations

• The O(m^6) dense Cholesky per factor makes the method costly when large conditioning sets are needed.
• Exact equivalence holds only for exact gradients, isotropic kernels with iid unscaled noise, or metric-matched noise; non-isotropic ARD with iid unscaled noise is not exact.
• Robust performance is shown only for small conditioning sets around m=20 on tested tasks.

Experimental setup

Datasets: MD22 GP simulations (Matern-5/2, Sobol inputs) Ackley-200D Levy-200D

Benchmarks: MD22 Ackley-200D Levy-200D GP simulations

Metrics: marginal KL divergence energy RMSE per atom simple regret wall-clock time peak GPU memory

Baselines: Exact GP Exact dGP Exact dGP-deRoos Vecchia dGP Vecchia dGP-deRoos DDSVGP DSoftKI Sobol VBO TuRBO-1

Builds on: Vecchia approximation de Roos et al. derivative GP decomposition DSoftKI DDSVGP GPyTorch

Closest papers by meaning

• Scaling Gaussian Processes with Derivative Information Using Variational Inference (NeurIPS 2021 · similarity 0.91)
• High-Dimensional Gaussian Process Inference with Derivatives (ICML 2021 · similarity 0.91)
• Scaling Gaussian Process Regression with Derivatives (NeurIPS 2018 · similarity 0.89)
• Spatio-Temporal Variational Gaussian Processes (NeurIPS 2021 · similarity 0.88)
• Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project (NeurIPS 2025 · similarity 0.87)
• Conditioning Gaussian Processes on Almost Anything (NeurIPS 2026 · similarity 0.87)