Research map / Probabilistic and scientific modeling / Variational and generative sampling

Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models

NeurIPS 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Hierarchical VAEs and posterior collapse · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Applies structured stochastic variational inference to GPLVMs so the latent-variable posterior depends on the sampled inducing variables, and reports better NLPD/RMSE/CRPS than mean-field amortized GPLVMs on Oilflow, qPCR, MNIST and FMNIST.

Problem

Mean-field variational approximations in GPLVMs decouple latent positions from the GP inducing variables, which tends to underestimate epistemic uncertainty of the learned data manifold.

Approach

Make the variational posterior over latent variables q(x_n | U) conditional on the inducing variables U (structured SVI), with amortized encoders that take the observation and a reparameterized sample of U, instead of the mean-field factorization q(x_n) q(U).

Key results

• On MNIST, A-S-GPLVM test RMSE 0.1537 vs 0.1568 for A-MF-GPLVM and test NLPD -348.4 vs -334.0, with significant differences (p<1e-4).
• On qPCR, A-S-GPLVM test NLPD 33.5 vs 38.7 for A-MF-GPLVM and test CRPS 0.2558 vs 0.3573.
• On Oilflow, IWVI-GPLVM obtains lower RMSE (test 0.0375 vs 0.0479) while A-S-GPLVM has better NLPD (-18.8 vs -17.4 for IWVI).

Surprising findings

• On Oilflow, the structured model improves NLPD and ELBO significantly while RMSE differences vs A-MF-GPLVM are not significant.
• Peak GPU memory rises from 29 MB to 599 MB (about 20x) for the structured variant, with wall-clock time up about 1.66x.

Limitations

• Substantial memory overhead from K Monte Carlo samples of inducing variables.
• Restrictive variational family: diagonal Gaussian q(x_n|U) and factorized q(U).
• Mixed results on the small datasets versus IWVI-GPLVM, with empirical coverage not uniformly improved.

Experimental setup

Datasets: Oilflow qPCR MNIST Fashion-MNIST

Benchmarks: Oilflow qPCR MNIST FMNIST

Metrics: RMSE NLPD negative ELBO CRPS empirical coverage Welch's t-test p-values

Baselines: MF-GPLVM A-MF-GPLVM IWVI-GPLVM

Builds on: Bayesian GPLVM Structured stochastic variational inference sparse variational GP amortized GPLVM importance-weighted variational inference

Closest papers by meaning

• Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties (NeurIPS 2020 · similarity 0.92)
• Variational Inference for Gaussian Process Models with Linear Complexity (NeurIPS 2017 · similarity 0.91)
• Pointwise uncertainty quantification for sparse variational Gaussian process regression with a Brownian motion prior (NeurIPS 2023 · similarity 0.91)
• Orthogonally Decoupled Variational Gaussian Processes (NeurIPS 2018 · similarity 0.89)
• New Bounds for Sparse Variational Gaussian Processes (ICML 2025 · similarity 0.88)
• Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling (ICML 2025 · similarity 0.87)