Research map / Probabilistic and scientific modeling / Variational and generative sampling
Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes two neighbour-driven approximations (HPA and SPA) to GP priors in GPVAEs, enabling mini-batch scalable latent GP inference without many inducing points.
Problem
Exact GP prior inference in GPVAEs costs O(N^3) and cannot be mini-batched, while inducing-point approximations need many inducing points for rapidly varying data and are hard to optimise.
Approach
Impose nearest-neighbour sparsity on the latent GP prior: HPA uses a hierarchical binary selection of top-H neighbours per mini-batch point (sparse covariance), while SPA chain-factorises the GP into conditionals on H nearest neighbours (sparse precision), both plugged into amortised VAE training.
Key results
- On rotated MNIST missing-frame generation, GPVAE-SPA-H10 reaches NLL 69.538 and RMSE 5.412, beating SVGPVAE-M30 (77.792 / 5.572) and MGPVAE (72.217 / 5.496).
- On MuJoCo conditional generation, GPVAE-HPA-H10 achieves NLL -2.335 and RMSE 0.0222 versus MGPVAE's -1.708 and 0.0398, with lower training time (37.0 vs 57.3 s/epoch).
- On SPE10 (~141,900 points), GPVAE-HPA-H20 and GPVAE-SPA-H20 reach RMSE about 0.638 versus 0.683 for VNNGP-H20 and 0.979 for SVGPVAE-M100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use neighbour-driven HPA or SPA instead of the factorised VAE prior | GPVAE · Rotated MNIST missing frames | NLL: 72.217 ± 0.192 -> 71.264 ± 1.263; 72.217 ± 0.192 -> 69.538 ± 0.664 | Helped |
| Use HPA or SPA with 10 neighbours instead of MGPVAE | GPVAE · MuJoCo Hopper conditional generation | NLL: -1.708 ± 0.057 -> -2.335 ± 0.032; -1.708 ± 0.057 -> -1.715 ± 0.159 | Helped |
| Use HPA or SPA with 20 neighbours instead of VNNGP-H20 | GPVAE · SPE10 imputation | RMSE: 0.683 ± 0.000 -> 0.638 ± 0.007; 0.683 ± 0.000 -> 0.638 ± 0.004 | Helped |
| Increase the number of HPA neighbours from 3 to 15 (ablation) | GPVAE-HPA · Moving Ball trajectory reconstruction | RMSE: 17.400 ± 3.691 -> 1.709 ± 0.146 | Helped |
| Increase the number of SPA neighbours from 3 to 10 (ablation) | GPVAE-SPA · Moving Ball trajectory reconstruction | RMSE: 3.507 ± 0.318 -> 1.079 ± 0.097 | Helped |
| Increase the number of SPA neighbours from 10 to 15 (ablation) | GPVAE-SPA · Moving Ball trajectory reconstruction | RMSE: 1.079 ± 0.097 -> 1.245 ± 0.063 | Hurt |
| Increase HPA neighbours from H=5 to H=20 (ablation) | GPVAE-HPA · Rotated MNIST missing frames | NLL: 101.741 ± 0.712 -> 69.637 ± 1.027 | Helped |
| Increase SPA neighbours from H=5 to H=20 (ablation) | GPVAE-SPA · Rotated MNIST missing frames | NLL: 70.865 ± 0.656 -> 69.516 ± 0.543 | Helped |
| Increase HPA neighbours from H=5 to H=20 (ablation) | GPVAE-HPA · MuJoCo Hopper conditional generation | RMSE: 0.0225 ± 0.001 -> 0.0221 ± 0.000 | Helped |
| Increase SPA neighbours from H=5 to H=20 (ablation) | GPVAE-SPA · MuJoCo Hopper conditional generation | RMSE: 0.0392 ± 0.011 -> 0.0339 ± 0.005 | Helped |
Surprising findings
- On corrupted-frame imputation, MGPVAE and GPVAE-Band reach lower RMSE than the proposed models, and GPVAE-Band has the longest training time.
- On the moving ball task, GPVAE-HPA with small H (3) performs very poorly (RMSE 17.4), and performance is non-monotone for SPA (best at H=10).
Limitations
- Only Euclidean distance was used for neighbour selection; alternative or manifold-aware metrics are unexplored.
- On corrupted-frame imputation, the proposed models do not beat MGPVAE or GPVAE-Band in RMSE/NLL.
- Neighbour search and Cholesky of H x H blocks add overhead; SPA uses same mini-batch for data and neighbour terms in practice rather than the theoretical separate batches.
Experimental setup
Datasets: Moving ball Rotated MNIST Healing MNIST MuJoCo Hopper (DeepMind Control Suite) Jura SPE10
Benchmarks: Moving ball Rotated MNIST MuJoCo Hopper Jura SPE10
Metrics: NLL RMSE training time (s/epoch)
Baselines: SVGPVAE LVAE MGPVAE SGPBAE VAE HI-VAE GPVAE-Diag GPVAE-Band VNNGP MOGP Exact GP SVGP
Builds on: GPVAE SVGPVAE NNGP VNNGP MGPVAE
Closest papers by meaning
- Gaussian Process Prior Variational Autoencoders (NeurIPS 2018 · similarity 0.92)
- Inter-domain Inference for Gaussian Process Variational Autoencoders (NeurIPS 2026 · similarity 0.92)
- Bayesian Basis Function Approximation for Scalable Gaussian Process Priors in Deep Generative Models (ICML 2025 · similarity 0.92)
- Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties (NeurIPS 2020 · similarity 0.90)
- High-Dimensional Bayesian Optimisation with Gaussian Process Prior Variational Autoencoders (ICLR 2025 · similarity 0.90)
- Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.