Research map / Multi-agent systems and games / LLM agents
Enhancing Complex Symbolic Logical Reasoning of Large Language Models via Sparse Multi-Agent Debate
Haocheng Yang, Fengxiang Cheng, Tianjun Yao, Mengyue Yang, Jiajun Chai, Xiaohan Wang, Guojun Yin, Wei Lin, Soummya Kar, Fenrong Liu, Haoxuan Li, Yisen Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a sparse multi-agent debate framework that combines symbolic and natural-language reasoning agents to improve complex symbolic logical reasoning in LLMs.
Problem
LLMs struggle with complex logical reasoning, and single-agent methods are limited by the capability of one model.
Approach
First multi-agent approach aimed specifically at LLM logical reasoning; it exploits the complementary strengths and weaknesses of symbolic and natural-language reasoning agents through sparse (rather than fully connected) debate.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adaptive sparse communication gate to the debate (ablation) | multi-agent debate (GPT-4) · ProofWriter | accuracy: 90.17% -> 92.00% | Improved |
| add adaptive sparse gate (Claude 3.7 Sonnet) (ablation) | multi-agent debate (Claude 3.7) · ProofWriter | accuracy: 97.00% -> 96.83% | No clear effect |
| add multi-agent translation debate (ablation) | MAD-Logic (GPT-4) · ProofWriter | accuracy: 89.17% -> 92.00% | Improved |
| add symbolic-solver reasoning agents to the debate (ablation) | MAD-Logic (GPT-4) · ProofWriter | accuracy: 79.33% -> 92.00% | Improved |
| add natural-language reasoning agents to the debate (ablation) | MAD-Logic (GPT-4) · ProofWriter | accuracy: 90.67% -> 92.00% | Improved |
| add NL reasoning agents to the debate (Claude 3.7) (ablation) | MAD-Logic (Claude 3.7) · ProofWriter | accuracy: 96.67% -> 96.83% | No clear effect |
| add more SL agents (FOL -> SAT+FOL+LP) (ablation) | MAD-Logic (GPT-4) · ProofWriter | accuracy: 85.50% -> 91.67% | Improved |
| add Plan-and-Solve NL agent to SAT+FOL+LP+CoT (ablation) | MAD-Logic (GPT-4) · ProofWriter | accuracy: 91.67% -> 92.00% | Improved |
| add debate rounds to 5 agents (vs one-shot direct vote) (ablation) | MAD-Logic (GPT-4) · ProofWriter | accuracy: 86.50% -> 92.00% | Improved |
| let SL and NL agents interact in debate (ablation) | MAD-Logic (GPT-4) · ProofWriter | accuracy: 90.83% -> 92.00% | Improved |
| use Simulate fallback instead of Random when solver fails (ablation) | MAD-Logic (GPT-4) · ProofWriter | accuracy: 89.83% -> 92.00% | Improved |
| use 3 SL translation agents instead of 1 (ablation) | translation stage · ProofWriter (GPT-4) | common error rate: 18.50% -> 0.33% | Improved |
Experimental setup
Builds on: multi-agent debate
Closest papers by meaning
- Multi-Agent Debate with Memory Masking (ICLR 2026 · similarity 0.91)
- Large Language Models Are Neurosymbolic Reasoners (AAAI 2024 · similarity 0.91)
- Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation (ICLR 2026 · similarity 0.91)
- When LLMs Develop Languages: Symbolic Communication for Efficient Multi-Agent Reasoning (ICML 2026 · similarity 0.90)
- MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models (ICML 2024 · similarity 0.90)
- Benefits and Limitations of Communication in Multi-Agent Reasoning (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
