Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Variational Polya Tree
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the variational Pólya tree (VPT), a Bayesian nonparametric prior over continuous densities trained via stochastic variational inference and plugged into flows/VAEs for density estimation, uncertainty and interpretability.
Problem
Deep density estimators lack interpretability and uncertainty quantification, and Bayesian nonparametric Pólya tree priors have been hard to combine with deep learning because MCMC inference does not scale.
Approach
Places a Pólya tree prior on the base distribution of normalizing flows and learns its Beta branching parameters by variational inference, using the conjugate joint posterior over tree nodes instead of a mean-field factorization, so the prior is trained end-to-end with stochastic gradients.
Key results
- On UCI tabular data, VPT with L=6 reports 162.72 log-likelihood on BSDS300 versus 156.33 for Block-NAF, and the authors claim state-of-the-art on most datasets except HEPMASS.
- On MNIST, the 4-level VPT reaches 0.94 negative log-likelihood, similar to RNODE's 0.97 (as reported in the paper).
- On CIFAR-10 with NICE, VPT (L=4) reaches 2.06 bits-per-dimension versus 2.57 for a learnable histogram prior.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VPT prior with Block-NAF rather than Block-NAF baseline | Block-NAF · POWER, GAS, HEPMASS, MINIBOONE, BSDS300 test sets | log-likelihood: 0.57 -> 0.67; 11.01 -> 11.92; -15.06 -> -15.01; -8.90 -> -8.71; 156.33 -> 158.59 | Helped |
| Increase VPT tree depth from L=4 to L=6 (ablation) | VPT with Block-NAF · POWER, GAS, HEPMASS, MINIBOONE, BSDS300 test sets | log-likelihood: 0.67 -> 0.61; 11.92 -> 12.20; -15.01 -> -13.94; -8.71 -> -8.51; 158.59 -> 162.72 | Mixed |
| Replace the VPT prior with a learnable histogram prior (ablation) | Block-NAF · POWER, GAS, BSD300 test sets | log-likelihood: 0.67 -> 0.62; 11.92 -> 11.00; 158.59 -> 156.98 | Hurt |
| Use VPT rather than MC-Dropout for predictive uncertainty | Uncertainty estimation · POWER, GAS, MNIST | SSE distance from 1: 1.30 -> 0.92; 1.35 -> 0.90; 1.27 -> 0.92 | Helped |
| Use VPT rather than mean-field Gaussian BNN weights for predictive uncertainty | Uncertainty estimation · POWER, GAS, MNIST | SSE distance from 1: 1.17 -> 0.92; 1.14 -> 0.90; 1.08 -> 0.92 | Helped |
| Use 4-level VPT rather than Gaussian-prior NICE | NICE · MNIST test | negative log-likelihood (bits-per-dimension): null | Helped |
| Increase VPT tree depth from L=2 to L=4 (ablation) | NICE · CIFAR-10 test | negative log-likelihood (bits-per-dimension): null | Helped |
| Replace the VPT prior with a learnable histogram prior (ablation) | NICE · CIFAR-10 test | bits-per-dimension: 2.06 -> 2.57 | Hurt |
| Use a 4-level VPT rather than RNODE | NICE density estimation · MNIST test | negative log-likelihood (bits-per-dimension): 0.94 vs 0.97 | Mixed |
Surprising findings
- The VPT adds negligible overhead: under 0.05% extra peak GPU memory and at most 1.3x training time per epoch.
- VPT slightly over-estimates predictive variance (SSE 0.90–0.92) while MC-dropout and BNN under-estimate it.
Limitations
- Computational cost scales as O(2^{LD}) with tree level and dimension, so deep trees over high-dimensional inputs are costly.
- Fixed tree level L may limit asymptotic consistency, as the authors acknowledge.
- Results on HEPMASS are not state-of-the-art (comparable to TAN); VPT's variance is slightly over-estimated.
Experimental setup
Datasets: POWER GAS HEPMASS MINIBOONE BSDS300 MNIST CIFAR-10 SVHN
Benchmarks: POWER GAS HEPMASS MINIBOONE BSDS300 MNIST CIFAR-10
Metrics: log-likelihood bits-per-dimension standardized squared error training time per epoch peak GPU memory
Baselines: Real NVP Glow MADE MoG FFJORD MAF MoG TAN NAF-DDSF Block-NAF NICE RNODE VDM MuLAN
Builds on: Pólya tree prior Block-NAF NICE Real NVP Mondrian forests
Closest papers by meaning
- TreeVI: Reparameterizable Tree-structured Variational Inference for Instance-level Correlation Capturing (NeurIPS 2024 · similarity 0.85)
- Forward $\chi^2$ Divergence Based Variational Importance Sampling (ICLR 2024 · similarity 0.85)
- SoftCVI: Contrastive variational inference with self-generated soft labels (ICLR 2025 · similarity 0.85)
- Variational Inference with Mixtures of Isotropic Gaussians (NeurIPS 2025 · similarity 0.84)
- Bayesian Online Natural Gradient (BONG) (NeurIPS 2024 · similarity 0.84)
- Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.