Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Scalable Gaussian Processes with Latent Kronecker Structure
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Latent Kronecker GPs (LKGP), which keep exact GP inference on partially observed gridded product-kernel data by projecting a latent Kronecker kernel matrix, combined with iterative solvers and pathwise conditioning.
Problem
Exact GP inference on large datasets is costly, and Kronecker-structured speedups for product kernels break when grid observations are missing, forcing approximations or generic iterative methods.
Approach
Express the kernel matrix over observed grid points as a projection P (K_S ⊗ K_T) Pᵀ of a full latent Kronecker product, so MVMs exploit Kronecker structure via zero-padding and slicing without any approximation of the GP prior.
Key results
- On LCBench (every fifth dataset, 10 seeds), LKGP achieves the best train RMSE and test NLL on average and the lowest runtime (e.g., ~0.37 min vs ~6.5 min for SVGP on APSFailure), though SVGP/CaGP have better test RMSE.
- On Nordic climate data with up to 5M points, LKGP outperforms SVGP, VNNGP, and CaGP in RMSE and NLL at all missing ratios (10%-50%) while requiring the least time (e.g., 28.7 min vs 150 min for SVGP at 10% missing on temperature).
- Empirical break-even points for time and memory against iterative methods without latent structure closely match the derived asymptotic break-even points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LKGP with latent Kronecker structure instead of CaGP | Climate temperature prediction · 10% missing, test | RMSE: 0.18 ± 0.00 -> 0.08 ± 0.00 | Helped |
| Use LKGP with latent Kronecker structure instead of CaGP | Climate temperature prediction · 10% missing, test | NLL: -0.22 ± 0.00 -> -1.14 ± 0.01 | Helped |
| Use LKGP with latent Kronecker structure instead of SVGP | Climate temperature prediction · 10% missing | time (min): 150 ± 0.08 -> 28.7 ± 0.33 | Helped |
| Use LKGP with latent Kronecker structure instead of VNNGP | Climate precipitation prediction · 10% missing, test | RMSE: 0.46 ± 0.00 -> 0.24 ± 0.00 | Helped |
| Use LKGP with latent Kronecker structure instead of CaGP | Climate precipitation prediction · 10% missing, test | NLL: 0.94 ± 0.00 -> -0.02 ± 0.00 | Helped |
| Use LKGP with latent Kronecker structure instead of SVGP | Climate precipitation prediction · 10% missing | time (min): 150 ± 0.02 -> 15.6 ± 0.01 | Helped |
Surprising findings
- LKGP has the best train metrics but worse test RMSE than SVGP and CaGP on LCBench, which the authors attribute to overfitting caused by non-uniform missingness at the end of learning curves.
Limitations
- Assumes a product kernel, so points are correlated only if close in both S and T.
- Requires data on a partial grid; non-grid data needs artificial grid generation such as interpolation.
- Less efficient than generic iterative methods when the missing ratio is high (beyond the break-even point).
Experimental setup
Datasets: SARCOS LCBench Nordic Gridded Climate Dataset
Benchmarks: SARCOS LCBench Nordic Gridded Climate Dataset
Metrics: RMSE NLL runtime memory usage
Baselines: SVGP VNNGP CaGP iterative methods without latent Kronecker structure
Builds on: Kronecker GP methods pathwise conditioning stochastic variational GP computation-aware GP GPyTorch
Closest papers by meaning
- Scalable Gaussian Process Separation for Kernels with a Non-Stationary Phase (ICML 2020 · similarity 0.88)
- Exact Gaussian Processes on a Million Data Points (NeurIPS 2019 · similarity 0.88)
- Implicit Gaussian process representation of vector fields over arbitrary latent manifolds (ICLR 2024 · similarity 0.85)
- Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference (NeurIPS 2024 · similarity 0.85)
- Sparse Gaussian Processes: Structured Approximations and Power-EP Revisited (NeurIPS 2025 · similarity 0.85)
- Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.