Research map / Probabilistic and scientific modeling / Variational and generative sampling
Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds DAG-shaped probabilistic integral circuits from region graphs, trains them by materializing tensorized quadrature PCs (QPCs), and scales training with neural functional sharing via multi-headed Fourier-feature MLPs.
Problem
Probabilistic integral circuits (continuous latent variable models) were limited to tree-shaped structures and their training via numerical quadrature was memory-intensive, so they could not scale to richer latent structures or large datasets.
Approach
A pipeline (RG -> PIC -> QPC -> folded QPC) that builds DAG-shaped PICs from arbitrary region graphs using Tucker- and CP-merge, materializes them into tensorized probabilistic circuits via quadrature, and shares neural functions (F-sharing and C-sharing with multi-headed MLPs) so that function evaluations and trainable parameters scale much better.
Key results
- QPCs outperform standard PCs on RGB datasets: e.g., CIFAR 5.09 bpd (QPC*) vs 5.50 (PC*) under YCoCg-R, and 4.48 vs 4.85 under YCoCg.
- Functional sharing lets PICs train with resources comparable to PCs; PICs without sharing go OOM for K > 64, while PICs with sharing have only ~6M+ trainable parameters vs hundreds of millions for PCs.
- The folded QG-CP-512 architecture with functional sharing uses about 99% fewer trainable parameters (about 387K MLP params vs 262M) to materialize the same tensor.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train a QPC distilled from a PIC instead of a standard PC | QG-CP-512 · CIFAR, YCoCg-R | bpd: 5.50 -> 5.09 | Helped |
| Train a QPC distilled from a PIC instead of a standard PC | QG-CP-512 · CIFAR, YCoCg | bpd: 4.85 -> 4.48 | Helped |
| Train a QPC distilled from a PIC instead of a standard PC | QG-CP-512 · ImgNet32, YCoCg-R | bpd: 5.25 -> 5.08 | Helped |
| Train a QPC distilled from a PIC instead of a standard PC | QG-CP-512 · ImgNet32, YCoCg | bpd: 4.63 -> 4.46 | Helped |
| Train a QPC distilled from a PIC instead of a standard PC | QG-CP-256 · ImgNet64, YCoCg-R | bpd: 5.22 -> 5.05 | Helped |
| Train a QPC distilled from a PIC instead of a standard PC | QG-CP-256 · ImgNet64, YCoCg | bpd: 4.59 -> 4.42 | Helped |
| Train a QPC distilled from a PIC instead of a standard PC | QG-CP-256 · CelebA, YCoCg-R | bpd: 4.78 -> 4.73 | Helped |
| Train a QPC distilled from a PIC instead of a standard PC | QG-CP-256 · CelebA, YCoCg | bpd: 4.16 -> 4.11 | Helped |
| Add C-sharing across integral-unit groups (ablation) | PIC training · QT-CP, K=128 | optimization-step time (ms): OOM -> 25 | Helped |
| Add C-sharing across integral-unit groups (ablation) | PIC training · QG-CP, K=64 | optimization-step time (ms): OOM -> 52 | Helped |
| Add C-sharing across integral-unit groups (ablation) | PIC training · QG-TK, K=32 | optimization-step time (ms): OOM -> 89 | Helped |
| Add C-sharing across integral-unit groups (ablation) | PIC training · QT-CP, K=128 | GPU memory (GiB): OOM -> 0.97 | Helped |
Surprising findings
- QPCs distilled from PICs, trained by gradient descent with Adam, outperform PCs trained with the same optimizer and are competitive with LVD-based PCs.
Limitations
- Sampling from PICs is not possible because differentiable sampling through multi-headed MLPs is not supported.
- Traditional continuous latent variable models such as VAEs, flows and diffusion models remain more scalable.
- Training is via numerical quadrature and maximum likelihood, which is memory- and time-intensive for large K.
Experimental setup
Datasets: MNIST FashionMNIST EMNIST CIFAR ImageNet32 ImageNet64 CelebA
Benchmarks: MNIST FashionMNIST EMNIST CIFAR ImageNet32 ImageNet64 CelebA
Metrics: bits per dimension (bpd) GPU memory time per optimization step number of trainable parameters
Baselines: PC (Adam/EM) HCLT SparsePC RAT-SPN IDF BitSwap BBans McBits LVD LVD-PG
Builds on: probabilistic integral circuits probabilistic circuits RAT-SPN EiNets HCLT
Closest papers by meaning
- Scaling Probabilistic Circuits via Monarch Matrices (ICML 2025 · similarity 0.86)
- Sum of Squares Circuits (AAAI 2025 · similarity 0.86)
- Scaling Tractable Probabilistic Circuits: A Systems Perspective (ICML 2024 · similarity 0.85)
- On the Hardness of Approximating Distributions with Tractable Probabilistic Models (NeurIPS 2025 · similarity 0.84)
- The Expressivity Boundary of Probabilistic Circuits: A Comparison with Large Language Models (NeurIPS 2026 · similarity 0.84)
- Probabilistic Neural Circuits (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.