Research map / Probabilistic and scientific modeling / Variational and generative sampling
Neural Network Approximators for Marginal MAP in Probabilistic Circuits
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSMP, a self-supervised neural network that approximates marginal MAP (and MPE) inference in probabilistic circuits using a differentiable multilinear loss computed via a query-specific PC, giving linear-time inference.
Problem
Marginal MAP (MMAP) and MPE inference are NP-hard in probabilistic circuits, while existing linear-time approximations (max-product, max-marginal/ML, sequential) give poor-quality solutions when many query variables are involved.
Approach
Replace binary query leaves with continuous NN outputs in [0,1], so the PC's value becomes a multilinear function of the query estimates; use its negative log as a self-supervised loss (with an entropy penalty to push outputs toward 0/1), trained on evidence samples drawn from the PC, so that a trained NN outputs MMAP solutions in one forward pass.
Key results
- On the 22 TPM datasets, SSMP beats Max in 88 of 154 MPE cases (64 losses) and 97 of 154 MMAP cases (46 losses).
- SSMP inference takes roughly 7-10 microseconds on an A40 GPU versus about 7 milliseconds for Max, roughly 1000x faster.
- On MNIST SSMP is best on 9 of 10 per-class PCs; on CIFAR-10 it dominates all competing methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SSMP instead of Max | PC inference · TPM datasets, MPE | log-likelihood: 88 wins; 64 losses | Helped |
| Use SSMP instead of ML | PC inference · TPM datasets, MPE | log-likelihood: 96 wins; 49 losses | Helped |
| Use SSMP instead of Seq | PC inference · TPM datasets, MPE | log-likelihood: 77 wins; 63 losses | Helped |
| Use SSMP instead of Max | PC inference · TPM datasets, MMAP | log-likelihood: 97 wins; 46 losses | Helped |
| Use SSMP instead of ML | PC inference · TPM datasets, MMAP | log-likelihood: 102 wins; 34 losses | Helped |
| Use SSMP instead of Seq | PC inference · TPM datasets, MMAP | log-likelihood: 82 wins; 53 losses | Helped |
| Initialize hill climbing with SSMP instead of Max | Stochastic hill climbing · TPM datasets, MPE | log-likelihood: 93 wins; 40 losses | Helped |
| Initialize hill climbing with SSMP instead of ML | Stochastic hill climbing · TPM datasets, MMAP | log-likelihood: 100 wins; 26 losses | Helped |
| Use SSMP instead of Max | PC inference · CIFAR-10, MPE | log-likelihood: 9 wins; 0 losses | Helped |
| Use SSMP instead of Max | PC inference · MNIST, MPE | log-likelihood: 9 wins; 1 loss | Mixed |
| Use SSMP instead of Max | PC inference · EMNIST, MPE | log-likelihood: 7 wins; 1 loss | Mixed |
| Use SSMP instead of Max | PC inference · TPM datasets, large query sets | log-likelihood | Mixed |
Surprising findings
- SSMP's advantage over Max grows with the query ratio and, for MMAP, with task difficulty, which the authors describe as improving as complexity increases.
- Supervised NN training on labels from stochastic hill climbing underperforms the self-supervised SSMP in nearly all cases.
Limitations
- Assumes evidence and query variable sets are fixed a priori; variable-length or arbitrary query sets are future work.
- Only binary variables are handled in the main method; multi-valued extension is claimed but not evaluated.
- Approximation quality is not compared to exact MMAP optimum at scale; many comparisons are win counts rather than gap to optimum.
Experimental setup
Datasets: 22 TPM datasets binarized MNIST EMNIST Letters binarized CIFAR-10
Benchmarks: 22 TPM datasets MNIST EMNIST CIFAR-10
Metrics: average log-likelihood of MPE/MMAP solution contingency win counts inference time
Baselines: Max approximation Maximum likelihood approximation Sequential approximation Stochastic hill climbing Supervised NN with MSE
Builds on: sum-product networks max-product inference stochastic hill climbing DeeProb-kit
Closest papers by meaning
- A Neural Network Approach for Efficiently Answering Most Probable Explanation Queries in Probabilistic Models (NeurIPS 2024 · similarity 0.87)
- On the Hardness of Approximating Distributions with Tractable Probabilistic Models (NeurIPS 2025 · similarity 0.87)
- Probabilistic Neural Circuits (AAAI 2024 · similarity 0.86)
- Learning to Condition: A Neural Heuristic for Scalable MPE Inference (NeurIPS 2025 · similarity 0.85)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.84)
- Generative Marginalization Models (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.