Research map / Probabilistic and scientific modeling / Molecules and proteins
LogicMP: A Neuro-symbolic Approach for Encoding First-order Logic Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LogicMP, a plug-in neural layer that performs efficient mean-field inference over a Markov Logic Network by reducing grounding messages and using Einstein summation, enabling first-order logic constraints in arbitrary neural networks.
Problem
Integrating first-order logic constraints into neural networks requires MLN inference over massive propositional groundings, which is #P-hard for exact inference and computationally expensive for approximate methods.
Approach
Derive a mean-field iteration whose grounding messages only consider true premises (for clauses) and aggregate messages per implication via Einstein summation with contraction-order optimization, so each MF iteration becomes parallel tensor operations usable as a differentiable neural layer.
Key results
- On Cora, ExpressGNN w/ LogicMP reaches 0.82 average AUC-PR versus 0.64 for ExpressGNN w/ GS, and on UW-CSE 0.30 versus 0.11.
- LogicMP runs joint inference over up to 262K coexistence variables on FUNSD within 0.03 seconds, where semantic-loss arithmetic-circuit compilation fails beyond sequence length 8.
- On CoNLL-2003 with list rules, LogicMP improves F1 on list-structured samples from 94.68 to 97.41.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LogicMP to ExpressGNN w/ GS training | ExpressGNN · Kinship | AUC-PR: .98 -> .99 | Helped |
| Add LogicMP to ExpressGNN w/ GS training | ExpressGNN · UW-CSE | AUC-PR: .11 -> .30 | Helped |
| Add LogicMP to ExpressGNN w/ GS training | ExpressGNN · Cora | AUC-PR: .64 -> .82 | Helped |
| Use LogicMP instead of ExpressGNN w/ GS for training | ExpressGNN · UW-CSE | training efficiency: about 10 times | Helped |
| Add LogicMP with the list rule | BLSTM · CoNLL-2003 samples with a list structure | F1: 94.68 to 97.41 | Helped |
| Add the transitivity FOLC using LogicMP | LayoutLM-Pair · FUNSD full and long | F1 | Helped |
| Combine LogicMP with SLrelax | LayoutLM-Pair · FUNSD | F1 | Helped |
Surprising findings
- Performance of LogicMP kept improving with more training, suggesting the reported ExpressGNN w/ GS results were limited by its training inefficiency.
Limitations
- Exact einsum path optimization is NP-hard, though practical for arity below 6.
- Rule-based mean-field only approximates MLN posterior; output is nearly optimal rather than exact.
- Semantic loss and SPL fail on the FUNSD task due to arithmetic circuit compilation, and SLrelax is an unrigorous relaxation.
Experimental setup
Datasets: FUNSD Kinship UW-CSE Cora CoNLL-2003 Smoke
Benchmarks: FUNSD Kinship UW-CSE Cora CoNLL-2003
Metrics: F1 AUC-PR
Baselines: LayoutLM SLrelax SPL SPADE SpanNER BIOES MCMC BP/Lifted BP MC-SAT HL-MRF ExpressGNN ExpressGNN w/ GS
Builds on: Markov Logic Networks CRFasRNN ExpressGNN Semantic Loss
Closest papers by meaning
- Efficient Probabilistic Logic Reasoning with Graph Neural Networks (ICLR 2020 · similarity 0.88)
- On Lifted Inference Using Neural Embeddings (AAAI 2019 · similarity 0.87)
- Neural Logic Machines (ICLR 2019 · similarity 0.87)
- Neuro-symbolic Learning Yielding Logical Constraints (NeurIPS 2023 · similarity 0.87)
- DL2: Training and Querying Neural Networks with Logic (ICML 2019 · similarity 0.86)
- Convolutional Differentiable Logic Gate Networks (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.