Research map / Probabilistic and scientific modeling / Variational and generative sampling
On the Relationship Between Monotone and Squared Probabilistic Circuits
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows monotone and squared probabilistic circuits are incomparable in expressive efficiency, and proposes Inception PCs (deep sum-of-square-of-sums with complex parameters) that generalize both, outperforming them on 20 binary datasets and downscaled ImageNet.
Problem
Monotone probabilistic circuits (non-negative weights) and squared probabilistic circuits (squared real/complex circuit functions) are each tractable density models, but their relative expressive efficiency is unclear; the paper asks whether they can be reconciled and improved.
Approach
Interpret monotone PCs as summing latent variables outside the square (sum-of-squares) and squared PCs as summing inside the square (square-of-sums); combine both into Inception PCs, a deep sum-of-square-of-sums with complex parameters, which materializes into a tractable structured-decomposable PC.
Key results
- Inception PCs achieve the best test negative log-likelihood on 14 of 20 binary datasets compared with monotone and squared PCs.
- On ImageNet32 the optimal Inception PC configuration (N1=32, N2=4) reaches 4.19 bpd versus 4.35 for monotone PC and 4.26 for squared PC.
- On ImageNet64 the Inception PC reaches 3.90 bpd versus 4.12 for monotone PC and 3.98 for squared PC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine 1-norm and 2-norm latents (N1=32, N2=4) under the stated computation budget | IncPC · ImageNet32 | bpd: 4.19 | Helped |
| use IncPC instead of MPC | probabilistic circuits · ImageNet32 | bpd: 4.35 -> 4.19 | Helped |
| use IncPC instead of SPC | probabilistic circuits · ImageNet64 | bpd: 3.98 -> 3.90 | Helped |
| combine both types of latents rather than use only monotone or only squared circuits (ablation) | IncPC · ImageNet32 | bpd: null | Helped |
| use complex rather than non-negative or real parameterization (ablation) | Inception PC · 20 binary datasets | test negative log-likelihood | No clear effect |
| use complex or negative parameterizations (ablation) | Inception PC · 20 binary datasets | test negative log-likelihood | Hurt |
| use Inception PCs rather than monotone or squared PCs | probabilistic circuits · 20 binary datasets | test negative log-likelihood | Helped |
| use squared PCs with non-negative parameters rather than monotone PCs at the same parameter count | probabilistic circuits · 20 binary datasets | test negative log-likelihood | Helped |
Surprising findings
- Monotone circuits can be exponentially more expressive efficient than squared circuits for a specific function class, so the two classes are incomparable.
- Complex and negative parameterizations often do not outperform non-negative ones on binary datasets, attributed partly to overfitting.
Limitations
- Training with complex parameters requires Wirtinger derivatives and is more complex to optimize.
- Inception PC complexity is higher than monotone or squared PCs per region (cubic in N2 and quadratic in N1).
- Complex and negative parameters appear more prone to overfitting on binary datasets.
Experimental setup
Datasets: 20 binary density estimation datasets (nltcs, msnbc, kdd-2k, plants, jester, audio, netflix, accidents, retail, pumsb-st ImageNet32 ImageNet64
Benchmarks: 20 binary datasets ImageNet32 ImageNet64
Metrics: test negative log-likelihood bits per dimension
Baselines: Monotone PC Squared PC RAT-SPN VAE EM-trained HCLT latent variable distillation
Builds on: probabilistic circuits squared probabilistic circuits latent variable interpretation of PCs hidden Chow-Liu trees RAT-SPN
Closest papers by meaning
- Sum of Squares Circuits (AAAI 2025 · similarity 0.88)
- Scaling Probabilistic Circuits via Monarch Matrices (ICML 2025 · similarity 0.86)
- On the Expressive Power of Tree-Structured Probabilistic Circuits (NeurIPS 2024 · similarity 0.85)
- Probabilistic Neural Circuits (AAAI 2024 · similarity 0.85)
- On the Hardness of Approximating Distributions with Tractable Probabilistic Models (NeurIPS 2025 · similarity 0.84)
- Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.