Research map / Probabilistic and scientific modeling / Variational and generative sampling
Probabilistic Neural Circuits
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces probabilistic neural circuits (PNCs), which relax decomposability by letting sum-unit weights be neural functions of ancestor variables, interpreted as deep mixtures of Bayesian networks, and evaluates them on MNIST-family density estimation and classification.
Problem
Probabilistic circuits offer tractable inference but are less expressive than neural networks because structural constraints (smoothness, decomposability) limit function approximation.
Approach
Generalize sum units of conditional probabilistic circuits so that mixture weights are neural networks conditioned on ancestor variables, yielding PNCs that approximate deep mixtures of Bayesian networks while remaining valid and partly tractable (ordered marginals and conditionals).
Key results
- PNCs achieve 0.87 bpd on MNIST and 2.51 bpd on FashionMNIST, beating PQC (1.20 and 3.47) and PSC (1.32 and 3.66) with about 2.8M parameters.
- On discriminative MNIST, PNC reaches 98.04% test accuracy versus 97.38% for PQC, but below LC (99.4%) and RAT-SPN (98.29%).
- PNCs outperform SHCLT, HCLT, CMC, RAT-SPN, IDF and BitSwap in bpd on all MNIST-family datasets reported.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use neural sum layers with data-dependent weights instead of conditional mixing operators | PNC versus PQC · MNIST family density estimation | bits per dimension: 1.20 -> 0.87; 3.47 -> 2.51; 1.84 -> 1.36; 1.83 -> 1.33; 1.86 -> 1.35; 1.76 -> 1.27 | Helped |
| Use neural sum layers with data-dependent weights instead of standard sum units | PNC versus PSC · MNIST family density estimation | bits per dimension: 1.32 -> 0.87; 3.66 -> 2.51; 2.07 -> 1.36; 2.07 -> 1.33; 2.16 -> 1.35; 2.02 -> 1.27 | Helped |
| Use PNC rather than SHCLT | MNIST family density estimation · MNIST family | bits per dimension: 1.14 -> 0.87; 3.27 -> 2.51; 1.52 -> 1.36; 1.58 -> 1.33; 1.60 -> 1.35; 1.54 -> 1.27 | Helped |
| Use PNC rather than PQC | Discriminative circuit learning · MNIST | accuracy: 97.38 -> 98.04 | Helped |
| Use PNC rather than PQC | Discriminative circuit learning · FashionMNIST | accuracy: 87.63 -> 88.84 | Helped |
| Use PNC rather than LC | Discriminative circuit learning · MNIST | accuracy: 99.4 -> 98.04 | Hurt |
| Use PNC rather than RAT-SPN | Discriminative circuit learning · FashionMNIST | accuracy: 89.89 -> 88.84 | Hurt |
Surprising findings
- Discriminative PNCs do not match logistic circuits or RAT-SPNs despite perfect training accuracy, which the authors attribute to lack of regularization.
- Relaxing decomposability trades tractability for better density estimation, with PNCs beating tractable circuit baselines.
Limitations
- Discriminative performance lags logistic circuits and RAT-SPNs, likely due to insufficient regularization.
- Only ordered (not arbitrary) marginals and conditionals are tractable; general marginal inference is lost.
- No structure learning method for PNCs; architecture is hand-designed for image grids.
Experimental setup
Datasets: MNIST FashionMNIST EMNIST
Benchmarks: MNIST FashionMNIST EMNIST
Metrics: bits per dimension classification accuracy
Baselines: PQC (CMO-SPQN) PSC SHCLT HCLT CMC RAT-SPN IDF BitSwap LC PQC
Builds on: probabilistic circuits sum-product-quotient networks conditional sum-product networks hidden Chow-Liu trees RAT-SPN
Closest papers by meaning
- On the Hardness of Approximating Distributions with Tractable Probabilistic Models (NeurIPS 2025 · similarity 0.87)
- The Expressivity Boundary of Probabilistic Circuits: A Comparison with Large Language Models (NeurIPS 2026 · similarity 0.87)
- On the Expressive Power of Tree-Structured Probabilistic Circuits (NeurIPS 2024 · similarity 0.86)
- A Unified Framework for Human-Allied Learning of Probabilistic Circuits (AAAI 2025 · similarity 0.86)
- Scaling Probabilistic Circuits via Monarch Matrices (ICML 2025 · similarity 0.86)
- Neural Network Approximators for Marginal MAP in Probabilistic Circuits (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.