Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Bayesian Basis Function Approximation for Scalable Gaussian Process Priors in Deep Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DGBFGP, a conditional deep generative model (GP-prior VAE) that replaces inducing points with a global Hilbert-space basis function approximation for mixed continuous/categorical additive GPs, giving linear-time training without amortized inference.
Problem
GP-prior VAEs for high-dimensional longitudinal data scale cubically with dataset size; inducing point optimization is hard for discrete covariates, and amortized variational posteriors introduce an amortization gap.
Approach
Use Hilbert-space (Laplacian eigenfunction) basis approximation for SE kernels and eigendecomposition of categorical kernels, combined into additive mixed-domain GP priors over global weights with hyperparameters treated probabilistically, trained by VI with mini-batch SGD and no per-sample encoder for latent GP values.
Key results
- On Rotated MNIST temporal interpolation, DGBFGP achieves MSE 0.009 ± 0.0001 vs 0.017 for L-NODE and 0.023 for SGP-BAE (latent dim 16).
- On Physionet, DGBFGP reaches MSE 0.619 ± 0.009 vs 0.718 for LVAE and 0.732 for BRITS.
- On SPRITES conditional generation, DGBFGP achieves MSE 0.0024 ± 0.0003 vs 0.0062 for LVAE and 0.0081 for SVGP-VAE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DGBFGP instead of L-NODE | Rotated MNIST temporal interpolation · Rotated MNIST test | MSE: 0.017 ± 0.0002 -> 0.009 ± 0.0001 | Helped |
| use DGBFGP instead of LVAE | Physionet forecasting · Physionet test subjects | MSE: 0.718 ± 0.007 -> 0.619 ± 0.009 | Helped |
| use DGBFGP instead of LVAE | SPRITES conditional generation · SPRITES unseen character combinations | MSE: 0.0062 ± 0.0004 -> 0.0024 ± 0.0003 | Helped |
| use semi-amortization (ablation) | DGBFGP, standard training · Rotated MNIST | MSE: 0.009 ± 0.0001 -> 0.010 ± 0.0004 | Hurt |
| use semi-amortization (ablation) | DGBFGP, standard training · Health MNIST | MSE: 0.009 ± 0.0000 -> 0.013 ± 0.0005 | Hurt |
| use semi-amortization (ablation) | DGBFGP, standard training · Physionet | MSE: 0.619 ± 0.009 -> 0.763 ± 0.007 | Hurt |
| exclude test-subject training samples and use semi-amortization (ablation) | DGBFGP · Rotated MNIST | MSE: 0.010 ± 0.0004 -> 0.013 ± 0.0004 | Hurt |
| exclude test-subject training samples and use semi-amortization (ablation) | DGBFGP · Health MNIST | MSE: 0.013 ± 0.0005 -> 0.015 ± 0.0006 | Hurt |
| exclude test-subject training samples and use semi-amortization (ablation) | DGBFGP · Physionet | MSE: 0.763 ± 0.007 -> 0.779 ± 0.003 | Hurt |
| exclude test-subject training samples and do not fine-tune (ablation) | DGBFGP · Rotated MNIST | MSE: 0.009 ± 0.0001 -> 0.071 ± 0.0002 | Hurt |
| exclude test-subject training samples and fine-tune with subject-specific parameters (ablation) | DGBFGP · Rotated MNIST | MSE: 0.013 ± 0.0004 -> 0.009 ± 0.0003 | Helped |
| exclude test-subject training samples and fine-tune with subject-specific parameters (ablation) | DGBFGP · Health MNIST | MSE: 0.015 ± 0.0006 -> 0.011 ± 0.0005 | Helped |
Surprising findings
- A non-fine-tuned global model (DGBFGP*) degrades sharply on unseen test subjects (e.g., Rotated MNIST MSE 0.071), while the semi-amortized SA-DGBFGP performs worse on Physionet than on simulated data.
Limitations
- Relies on fixed number of basis functions (M=10) for continuous kernels; performance depends on M
- Global parameterization without amortization generalizes poorly to unseen subjects unless fine-tuned with per-subject parameters
- Only stationary continuous kernels (SE) are demonstrated in experiments
Experimental setup
Datasets: Rotated MNIST Health MNIST Physionet Challenge 2012 SPRITES
Benchmarks: Rotated MNIST Health MNIST Physionet 2012 SPRITES
Metrics: MSE Sobol indices training time
Baselines: SVGP-VAE LVAE SGP-BAE L-NODE BRITS CVAE
Builds on: GP prior VAE Hilbert space GP approximation (Solin & Särkkä) LVAE SVGP-VAE LMM-VAE
Closest papers by meaning
- Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling (ICML 2025 · similarity 0.92)
- Gaussian Process Modeling of Approximate Inference Errors for Variational Autoencoders (CVPR 2022 · similarity 0.91)
- Gaussian Process Prior Variational Autoencoders (NeurIPS 2018 · similarity 0.91)
- High-Dimensional Bayesian Optimisation with Gaussian Process Prior Variational Autoencoders (ICLR 2025 · similarity 0.91)
- Inter-domain Inference for Gaussian Process Variational Autoencoders (NeurIPS 2026 · similarity 0.90)
- PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.