Research map / Large language models / Agents, code and math
When Form Changes but Logic Doesn’t: Building Logic-invariant LLMs through Structures
Xuyuan Liu, Xinshuai Dong, Elynn Chen, Yujun Yan
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The paper studies whether LLMs treat logically equivalent inputs consistently and proposes structure-based methods to build logic-invariant LLMs (abstract truncated, details unavailable).
Problem
Unclear whether LLM reasoning performance reflects understanding of underlying logical structure or reliance on shallow semantic cues; a key test is whether models give consistent predictions on logically equivalent inputs with different surface forms.
Approach
Based on the abstract only: frames logic invariance (same prediction for logically equivalent inputs) as an indicator of logical understanding and aims to build LLMs that are invariant to surface form through structural approaches. Specific mechanism not stated in the available text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add LoGIST logic embedding to GRPO training | Llama-3.1-8B-Ins GRPO · Synthetic logical equivalence, Mixed | logic inconsistency rate: 37.1% -> 16.6% | Improved |
| add LoGIST logic embedding to PPO training | Llama-3.1-8B-Ins PPO · Synthetic logical equivalence, Mixed | logic inconsistency rate: 30.6% -> 13.9% | Improved |
| add LoGIST logic embedding to SFT | Llama-3.1-8B-Ins SFT · Synthetic logical equivalence, Mixed | logic inconsistency rate: 19.0% -> 15.8% | Improved |
| add LoGIST to GRPO | Llama-3.1-8B-Ins GRPO · Synthetic Mat.Impl. (OOD form) | accuracy: 0.608 -> 0.654 | Improved |
| add LoGIST to PPO | Llama-3.1-8B-Ins PPO · Synthetic Base (in-distribution) | accuracy: 0.710 -> 0.708 | No clear effect |
| add LoGIST to SFT, PPO, GRPO | Llama-3.1-8B-Ins · ProverQA | accuracy: SFT 0.598 -> 0.638; PPO 0.568 -> 0.602; GRPO 0.626 -> 0.652 | Improved |
| add LoGIST to PPO and GRPO | Qwen2.5-3B-Ins · FOLIO | accuracy: PPO 0.596 -> 0.606; GRPO 0.621 -> 0.631 | Improved |
| prompt the LLM with replacement rules and formulas (ablation) | Qwen3-Next-80B · Mixed logical equivalence | accuracy (%): 66.9 -> 66.1 | No clear effect |
| give ROBDD as linearized prompt text instead of learned embedding (ablation) | SFT Llama-3.1-8B-Ins · Mixed logical equivalence | accuracy: 0.635 -> 0.639 | No clear effect |
| give ROBDD as linearized prompt text (ablation) | SFT Llama-3.1-8B-Ins · Mixed logical equivalence | logic inconsistency rate: 19.0% -> 21.3% | Hurt |
| add contrastive pretraining of the logic encoder (ablation) | LoGIST with GRPO on Qwen2.5-3B-Ins · Synthetic Mat.Impl. | accuracy: 0.571 -> 0.604 | Improved |
| build ROBDD from noisy parsed FOL instead of gold FOL (ablation) | GRPO + LoGIST Llama-3.1-8B-Ins · Synthetic Mixed | accuracy: 0.668 -> 0.656 | No clear effect |
Limitations
- Full text unavailable; only the truncated abstract was available
Closest papers by meaning
- RULEBREAKERS: Challenging LLMs at the Crossroads between Formal Logic and Human-like Reasoning (ICML 2025 · similarity 0.90)
- A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoning (NeurIPS 2025 · similarity 0.90)
- RATT: A Thought Structure for Coherent and Correct LLM Reasoning (AAAI 2025 · similarity 0.89)
- Logically Consistent Language Models via Neuro-Symbolic Integration (ICLR 2025 · similarity 0.89)
- Inductive Learning of Logical Theories with LLMs: A Expressivity-graded Analysis (AAAI 2025 · similarity 0.89)
- Content-free Logical Modification of Large Language Model by Disentangling and Modifying Logic Representation (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
