Research map / Large language models / Reasoning and chain of thought

Break the Block: Dynamic-size Reasoning Blocks for Diffusion Large Language Models via Monotonic Entropy Descent with Reinforcement Learning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes b1, an RL post-training framework that learns dynamic-size reasoning blocks for diffusion LLMs via a block-entropy monotonic-descent reward and a block-ending indicator reward, plugged into existing dLLM GRPO variants.

Problem

Fixed-size block semi-autoregressive decoding in diffusion LLMs disrupts reasoning steps and coherence, and the optimal block size varies across reasoning tasks.

Approach

Learn a block-ending indicator token via RL so blocks align with reasoning steps, and reward monotonic descent of block-wise mean token entropy (surrogate for negative Spearman correlation) as a plug-and-play addition to diffusion GRPO methods.

Key results

• On Countdown with 256 tokens, wd1 + b1 reaches 58.98% vs 39.45% for wd1 (+19.53 points).
• d1 + b1 improves GSM8K 256-token accuracy from 77.03 to 78.24 and MATH500 512-token accuracy to 38.60 vs 37.80.
• Ablation on wd1 Countdown: removing the MED reward drops accuracy from 58.98 to 44.14.

Surprising findings

• AdaBlock-dLLM, a tuning-free inference-time method, does not improve reasoning performance in the zero-shot setting and sometimes underperforms the base model.
• Without controlling the number of block indicators, the model reward-hacks by inserting indicators repeatedly, fragmenting reasoning.

Limitations

• Relies on a pre-defined target block count K_target and reward weights set to 1 without extensive tuning across tasks.
• The entropy surrogate averages marginal token entropies and does not model intra-block dependencies, as the authors acknowledge.
• Evaluated mainly on LLaDA-8B-Instruct with 4-bit LoRA; other dLLM backbones such as Dream are excluded.

Experimental setup

Datasets: GSM8K MATH Countdown Sudoku KodCodeLight-RL-10K

Benchmarks: GSM8K MATH500 Countdown Sudoku HumanEval MBPP KodCode MMLU MMLU-Pro ARC-E ARC-C

Metrics: zero-shot pass@1 accuracy average Spearman-based negative correlation r_SCC proportion of positive r_SCC blocks throughput tokens/s

Baselines: Diffu-GRPO GDPO d1 wd1 AdaBlock-dLLM SFT

Builds on: d1 wd1 GDPO GRPO LLaDA

Closest papers by meaning

• d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning (NeurIPS 2025 · similarity 0.93)
• Block-R1: Rethinking the Role of Block Size in Multi-domain Reinforcement Learning for Diffusion Large Language Models (NeurIPS 2026 · similarity 0.91)
• Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models (ICLR 2026 · similarity 0.90)
• T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling (ICML 2025 · similarity 0.90)
• Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy (ICLR 2026 · similarity 0.90)
• The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models (ICML 2026 · similarity 0.89)