Research map / Large language models / Reasoning and chain of thought
Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RaLU, a training-free test-time scaling framework that decomposes an LLM-generated program into control-flow logic units and iteratively judges, explains, and repairs each unit with rewind-and-correct before synthesizing a final solution.
Problem
LLM reasoning via CoT and program-aided methods suffers from "reasoning hallucinations": inconsistencies between NL reasoning steps and the logic of generated programs, hurting math and code reasoning.
Approach
Use static analysis (CFG) to split a generated program into logic units, then run a stateful multi-turn dialogue where the same LLM judges OK/WRONG, explains, and repairs each unit, rewinding to re-validate prior units after a fix; final solution is synthesized from the verified path.
Key results
- RaLU with DeepSeek-V3 reaches 0.971 on GSM8K and 0.821 on MATH-np, exceeding the best listed baselines and GPT-4o (0.697 on MATH-np).
- On Llama3.3-70B, RaLU improves HumanEval from 0.713 to 0.811 (+13.745% relative over best baseline) and HumanEval+ from 0.671 to 0.768.
- RaLU with DeepSeek-V3 scores 0.833 on MBPP+, above o1-preview's reported 0.802, though it is slightly below o1-preview on original HumanEval (0.939 vs 0.963).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RaLU instead of the strongest baseline | DeepSeek V3 · GSM8K | accuracy: +0.310 | Helped |
| Use RaLU instead of the strongest baseline | DeepSeek V3 · MATH-np | accuracy: +0.860 | Helped |
| Use RaLU instead of the strongest baseline | DeepSeek V3 · HumanEval | pass@1: +0.643 | Helped |
| Use RaLU instead of the strongest baseline | DeepSeek V3 · HumanEval+ | pass@1: +2.036 | Helped |
| Use RaLU instead of the strongest baseline | Llama3.3-72B · Mbpp | pass@1: +0.598 | Helped |
| Use RaLU instead of the strongest baseline | Qwen2.5-72B · Mbpp+ | pass@1: +4.678 | Helped |
| Replace CFG-driven units with line-by-line units (ablation) | RaLU on Llama3.3 · All benchmarks | performance: 7.04% decline | Hurt |
| Replace CFG-driven units with line-by-line units (ablation) | RaLU on Llama3.3 · All benchmarks | token consumption: 37.7% increase | Hurt |
| Replace program-guided logic units with CoT natural-language steps (ablation) | RaLU · Mathematical tasks | accuracy: 5.52% accuracy drop | Hurt |
| Replace program-guided logic units with CoT natural-language steps (ablation) | RaLU · Code reasoning | score: 4.35% score drop | Hurt |
Surprising findings
- Line-by-line decomposition of the program hurts performance by 7.04% and increases token use by 37.7% relative to CFG-based units.
- Replacing program-guided logic units with CoT natural-language steps reduces performance by 5.52% on math and 4.35% on code.
Limitations
- Evaluated only on math and code benchmarks; claimed generality to other domains is untested.
- Uses MATH-np subset due to resource limits rather than full MATH.
- Relies on the same LLM as judge and repairer, and the theoretical improvement condition depends on judge accuracy and repair rate.
Experimental setup
Datasets: GSM8K MATH-np HumanEval HumanEval+ MBPP MBPP+
Benchmarks: GSM8K MATH-np HumanEval HumanEval+ MBPP MBPP+
Metrics: accuracy pass@1
Baselines: Direct prompting Zero-Shot CoT Tree-of-Thought Self-Consistency Self-Calibration Self-Refine Program-of-Thought Self-Check Self-Debugging
Builds on: Chain-of-Thought Program-of-Thought Self-Consistency Self-Refine Self-Debugging
Closest papers by meaning
- Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.91)
- Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation (ICLR 2025 · similarity 0.91)
- GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (ICLR 2025 · similarity 0.91)
- Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus (NeurIPS 2024 · similarity 0.91)
- Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models (ICLR 2026 · similarity 0.90)
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.