Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
On the Independence Assumption in Neurosymbolic Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows theoretically that the conditional independence assumption in probabilistic neurosymbolic learning biases semantic-loss minima toward deterministic solutions and makes the optimisation landscape non-convex and disconnected.
Problem
Probabilistic neurosymbolic learning systems commonly assume symbols are conditionally independent given the input; this may bias learning toward overconfident predictions and hinder optimisation under logical constraints.
Approach
Characterises the set of possible independent distributions that minimise the semantic loss using implicants, cubical sets and computational homology, proving the independence assumption forces determinism on implicant variables and yields non-convex, disconnected minima, while expressive (joint) distributions give convex loss.
Key results
- Theorem: an independent distribution minimises the semantic loss iff it is deterministic on some variables forming an implicant of the constraint.
- Semantic loss is convex over independent distributions only when there is exactly one prime implicant; otherwise minima are non-convex.
- For the traffic light example, gradient descent on the independent model converges to deterministic minima lying on two line segments, whereas the expressive softmax model's minima spread over the feasible triangle but are biased toward vertices.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace conditionally independent distributions with expressive joint distributions | semantic loss on the traffic-light constraint · traffic-light constraint | ability to represent uncertainty over valid worlds | Helped |
| use conditionally independent distributions | semantic loss · traffic-light constraint | ability to represent uncertainty over valid worlds | Hurt |
| use conditionally independent distributions rather than expressive distributions | semantic loss · constraints with multiple prime implicants | loss convexity | Hurt |
| use conditionally independent distributions rather than expressive distributions | semantic loss · constraints with disconnected prime implicant graphs | connectedness of minima | Hurt |
| add entropy maximisation regularisation to the independent model (ablation) | independent model with semantic loss · traffic-light constraint | uncertainty calibration | No clear effect |
| add entropy maximisation regularisation with α=0.1 (ablation) | independent model with semantic loss · traffic-light constraint | probability assigned to the impossible world: 0.023 | Hurt |
| add entropy maximisation regularisation with α=0.01 (ablation) | expressive model with semantic loss · traffic-light constraint | distribution of valid-world probability | Helped |
| replace the product or Gödel t-conorm with the Łukasiewicz t-conorm (ablation) | fuzzy neurosymbolic loss · traffic-light constraint | bias toward deterministic choices | Helped |
| use the Łukasiewicz t-conorm (ablation) | fuzzy neurosymbolic loss · traffic-light constraint | convergence to solutions where the constraint is certain | Hurt |
| use a mixture of at least two independent components (ablation) | mixture of independent distributions · any constraint | connectedness of possible minima | Helped |
Surprising findings
- Using two mixture components of independent distributions makes the set of possible distributions connected, while the number of components needed to cover all possible worlds can be exponential in the number of variables.
- Entropy regularisation did not help calibrate the independent model and biased it further toward a single vertex.
Limitations
- Empirical evaluation limited to a toy traffic light example and visualisations; no large-scale benchmarks.
- Results on expressive distributions rely on exponentially many prime implicants and inference over joint distributions is harder.
Experimental setup
Datasets: MNIST Addition (illustrative)
Metrics: semantic loss
Baselines: semantic loss DeepProbLog DeepStochLog A-NeSI NeurASP Scallop pseudo-semantic loss NeuPSL BEARS
Builds on: semantic loss DeepProbLog Quine-McCluskey algorithm BEARS
Closest papers by meaning
- On the Hardness of Probabilistic Neurosymbolic Learning (ICML 2024 · similarity 0.91)
- EM-NeSy: Expectation Maximization for Neurosymbolic Learning (NeurIPS 2026 · similarity 0.86)
- Right for the Right Reasons: Avoiding Reasoning Shortcuts via Prototypical Neurosymbolic AI (NeurIPS 2025 · similarity 0.85)
- Neurosymbolic Diffusion Models (NeurIPS 2025 · similarity 0.85)
- Deciphering Raw Data in Neuro-Symbolic Learning with Provable Guarantees (AAAI 2024 · similarity 0.85)
- Neural Conditional Probability for Uncertainty Quantification (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.