Research map / Large language models / Agents, code and math
Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification
Chuxue Cao, Jinluan Yang, Haoran Li, Kunhao Pan, Zijian Zhao, Zhengyu Chen, Yuchen Tian, Lijun Wu, Conghui He, Yike Guo, Sirui Han
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add formal logic verification to SFT data (FLV-SFT vs Natural-SFT) (ablation) | Qwen2.5-7B SFT · six-benchmark average | average accuracy: 36.3 -> 49.8 | Improved |
| add formal logic verification to SFT data (ablation) | Qwen2.5-7B SFT · MATH500 | accuracy: 56.6 -> 77.2 | Improved |
| add GRPO with composite reward after FLV-SFT (ablation) | FLV-SFT Qwen2.5-7B · six-benchmark average | average accuracy: 49.8 -> 51.9 | Improved |
| add GRPO after FLV-SFT (ablation) | FLV-SFT Qwen2.5-14B · six-benchmark average | average accuracy: 55.7 -> 58.6 | Improved |
| add RL to natural-language SFT model (no formal verification) (ablation) | Natural-SFT Qwen2.5-7B · six-benchmark average | average accuracy: 36.3 -> 37.0 | No clear effect |
| skip SFT cold start, apply RL directly to base model (FLV-ZeroRL) (ablation) | FLV-RL Qwen2.5-7B · six-benchmark average | average accuracy: 51.9 -> 42.7 | Hurt |
| replace enforced per-step verification with flexible verification (decouple calculation from validation) (ablation) | FLV training data · math, logic, general subsets | accuracy | Improved |
| formal logic verification-guided RL vs tool-integrated SimpleTIR | Qwen2.5-7B · KOR-Bench | accuracy: 37.0 -> 51.0 | Improved |
| formal logic verification-guided RL vs tool-integrated SimpleTIR | Qwen2.5-7B · AIME24 | avg@16 accuracy: 41.0 -> 20.8 | Hurt |
| formal logic verification-guided RL vs General-Reasoner | Qwen2.5-7B · GPQA-Diamond | accuracy: 38.9 -> 35.4 | Hurt |
Closest papers by meaning
- Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation (ICLR 2025 · similarity 0.90)
- Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus (NeurIPS 2024 · similarity 0.90)
- ORACLE: Optimizing Reasoning Abilities of Large Language Models via Constraint-Led Synthetic Data Elicitation (AAAI 2026 · similarity 0.90)
- Variation in Verification: Understanding Verification Dynamics in Large Language Models (ICLR 2026 · similarity 0.90)
- Divide and Translate: Compositional First-Order Logic Translation and Verification for Complex Logical Reasoning (ICLR 2025 · similarity 0.89)
- APOLLO: Automated LLM and Lean Collaboration for Advanced Formal Reasoning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
