Research map / Large language models / Reasoning and chain of thought
DiffuReason: Enhancing Reasoning Ability for Diffusion Language Models via Monte Carlo Tree Search
YIPING SONG, Jinyu You, Zhiliang Tian, Jinsong Su, Minlie Huang, Chenping Hou
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffuReason, combining Monte Carlo Tree Search with diffusion language models to improve their reasoning accuracy while keeping the parallel-generation speed advantage over autoregressive MCTS.
Problem
Diffusion LLMs offer fast parallel generation but lag in accuracy on complex reasoning, while AR models with MCTS reason well but are slow due to serial token generation.
Approach
Applies Monte Carlo Tree Search to diffusion language models to add System 2-style search on top of parallel dLLM generation, aiming to get reasoning gains with lower latency than AR+MCTS. Details beyond the abstract are not available.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add lookahead simulation (ablation) | DiffuReason (WeDLM) · MATH500 | accuracy (%): 64.6 -> 68.2 | Improved |
| add lookahead simulation (ablation) | DiffuReason (WeDLM) · Gaokao2023 | accuracy (%): 46.4 -> 51.2 | Improved |
| add revision (ablation) | DiffuReason (WeDLM) · MATH500 | accuracy (%): 63.0 -> 68.2 | Improved |
| add revision (ablation) | DiffuReason (WeDLM) · Gaokao2023 | accuracy (%): 48.1 -> 51.2 | Improved |
| use block-wise sequential search instead of random node search (ablation) | DiffuReason (LLaDA) · MATH500 subset (100 problems) | overall accuracy (%): 9.4 -> 37.0 | Improved |
| use fast generation (32 steps) instead of fine generation (128 steps) (ablation) | DiffuReason (LLaDA) · MATH500 subset (100 problems) | overall accuracy (%): 39.0 -> 37.0, time 136.7 s -> 86.9 s | Mixed |
| use fast generation (32 steps) instead of 128 steps (ablation) | DiffuReason (LLaDA) · MATH500 Level 5 | accuracy (%): 10.0 -> 5.0 | Hurt |
| use block length 64 instead of 32 (ablation) | DiffuReason (LLaDA) · MATH500 subset | accuracy (%): 28.0 -> 37.0 | Improved |
| use block length 64 instead of 128 (ablation) | DiffuReason (LLaDA) · MATH500 subset | accuracy (%): 30.0 -> 37.0 | Improved |
| apply DiffuReason search to LLaDA-8B | LLaDA-8B · AIME2024 | accuracy (%): 6.7 -> 10.0 | Improved |
| apply DiffuReason search to WeDLM-8B | WeDLM-8B · AIME2024 | accuracy (%): 3.3 -> 20.0 | Improved |
| apply DiffuReason search to WeDLM-8B | WeDLM-8B · MATH500 | accuracy (%): 58.8 -> 68.2 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- WeDLM-8B — WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference (ICML 2026) · better on 4 of 4 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Experimental setup
Baselines: auto-regressive MCTS
Builds on: Monte Carlo Tree Search
Closest papers by meaning
- On the Reasoning Abilities of Masked Diffusion Language Models (ICLR 2026 · similarity 0.91)
- MRO: Enhancing Reasoning in Diffusion Language Models via Multi-Reward Optimization (NeurIPS 2025 · similarity 0.90)
- d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation (ICML 2026 · similarity 0.90)
- Theoretical Benefit and Limitation of Diffusion Language Model (NeurIPS 2025 · similarity 0.90)
- From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs (NeurIPS 2026 · similarity 0.89)
- Breaking AR’s Sampling Bottleneck: Provable Acceleration via Diffusion Language Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
