Research map / Label-efficient and robust learning / Domain adaptation and generalization
A Unified Framework for Human-Allied Learning of Probabilistic Circuits
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a unified framework that encodes diverse domain knowledge (generalization, CSI, monotonic, class-imbalance, privileged info, etc.) as differentiable equality/inequality constraints and learns PC parameters via a penalty method.
Problem
Parameter learning of probabilistic circuits is data-driven and overfits in data-scarce, knowledge-rich domains such as healthcare; existing knowledge incorporation into PCs is hard/exact or limited to equality constraints or specific PC subclasses.
Approach
Express diverse domain knowledge as differentiable equality or inequality constraints on marginal/conditional PC queries, and learn parameters by maximizing likelihood minus a penalty on constraint violation, optimized with an increasing penalty-weight schedule (penalty method), applicable to any PC whose violations are differentiable.
Key results
- On the BN datasets, RatSPN with CSI constraints improved test log-likelihood over the data-only baseline (e.g., earthquake -272.0 to -121.8; asia -483.3 to -313.2).
- With a generalization constraint on Set-MNIST-Even, EinsumNet+GC improved test log-likelihood from -666.2 to -561.9 and reduced variance markedly.
- On numom2b-a, RatSPN with knowledge reached -7288.2 versus -14573.9 for data-only RatSPN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CSI constraints | RatSPN · Asia, Sachs, Survey, Earthquake | test log-likelihood: -483.3 ± 4.1 -> -313.2 ± 3.9; -1097.5 ± 8.8 -> -861.2 ± 8.7; -611.7 ± 7.2 -> -476.6 ± 6.6; -272.0 ± 2.4 -> -121.8 ± 2.1 | Helped |
| Add monotonic influence constraints | RatSPN · Breast-cancer, Diabetes, Thyroid, Heart-disease | test log-likelihood: -2110.8 ± 15.6 -> -1271.5 ± 14.6; -7010.3 ± 31.0 -> -5070.3 ± 481.8; -351.5 ± 6.1 -> -200.5 ± 23.2; -931.7 ± 15.0 -> -73 | Helped |
| Add domain constraints | RatSPN · numom2b-a | test log-likelihood: -14573.9 ± 69.9 -> -7288.2 ± 1.6 | Helped |
| Add generalization constraint (GC) | RatSPN and EinsumNet · Helix; Set-MNIST-Even; Set-MNIST-Odd; Set-MNIST-Full; Set-Fashion-MNIST | test log-likelihood: RatSPN: -7.1 ± 1.8 -> -3.6 ± 0.1; -657.7 ± 15.9 -> -562.2 ± 0.4; -682.3 ± 16.7 -> -555.1 ± 0.4; -599.3 ± 3.2 -> -566.1 ± | Helped |
| Add a CSI, then add a monotonic influence statement (MIS) | RatSPN · Earthquake, Survey, Asia, numom2b-b | test log-likelihood: Earthquake: -137.7 ± 4.7 -> -106.1 ± 1.1; Survey: -523.5 ± 4.3 -> -470.9 ± 6.6; Asia: -320.5 ± 9.9 -> -284.7 ± 6.4; numo | Helped |
| Increase the constraint domain-set size (ablation) | EinsumNet+GC · Set-MNIST-Even | test log-likelihood: null | Helped |
| Add noise to the constraint domain set, up to 40% (ablation) | EinsumNet+GC · Set-MNIST-Even | test log-likelihood: null | No clear effect |
| Use a very small penalty weight λ (ablation) | EinsumNet+GC · Set-MNIST-Even | test log-likelihood: null | Hurt |
| Use a very large penalty weight λ (ablation) | EinsumNet+GC · Set-MNIST-Even | test log-likelihood: null | Hurt |
| Set penalty weight λ around 1 (ablation) | EinsumNet+GC · Set-MNIST-Even | test log-likelihood: null | Helped |
| Append GC data points to training data instead of using GC as a training constraint | RatSPN and EinsumNet · Helix | test log-likelihood: null | No clear effect |
Surprising findings
- Adding the same extra data points directly to training without structured GC integration did not improve generalization.
Limitations
- Limited to propositional logical constraints and fixed PC structure (parameter learning only).
- Sensitivity to penalty weight: too small underuses constraints, too large overwhelms likelihood.
Experimental setup
Datasets: Earthquake BN Asia BN Sachs BN Survey BN 3D Helix manifold MNIST Fashion-MNIST UCI breast-cancer UCI diabetes UCI thyroid UCI heart-disease nuMoM2b
Benchmarks: Earthquake Asia Sachs Survey 3D Helix Set-MNIST-Full Set-MNIST-Even Set-MNIST-Odd Set-Fashion-MNIST UCI breast-cancer UCI diabetes UCI thyroid
Metrics: test log-likelihood constraint violation
Baselines: RatSPN EinsumNet
Builds on: RatSPN EinsumNet penalty method
Closest papers by meaning
- Probabilistic Neural Circuits (AAAI 2024 · similarity 0.86)
- Scaling Probabilistic Circuits via Monarch Matrices (ICML 2025 · similarity 0.84)
- On the Expressive Power of Tree-Structured Probabilistic Circuits (NeurIPS 2024 · similarity 0.84)
- The Expressivity Boundary of Probabilistic Circuits: A Comparison with Large Language Models (NeurIPS 2026 · similarity 0.84)
- On the Hardness of Approximating Distributions with Tractable Probabilistic Models (NeurIPS 2025 · similarity 0.82)
- The Computational Complexity of Circuit Discovery for Inner Interpretability (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.