AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Lookahead Unmasking Elicits Reliable Decoding in Diffusion Language Models

Sanghyun Lee, Seungryong Kim, Jongho Park, Dongmin Park

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Lookahead Unmasking (LookUM), an inference-time method that chooses which tokens to unmask in masked diffusion language models by looking ahead rather than greedily using local confidence.

Problem

Masked diffusion language models depend heavily on the order in which tokens are unmasked at inference; confidence-based greedy ordering is short-sighted and lets early decoding errors cascade.

Approach

Replace local confidence-based unmasking order with a lookahead procedure that accounts for downstream consequences of unmasking choices, so that early decoding errors are avoided and extra test-time computation is used.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add LookUM lookahead path selectionLLaDA-8B · HumanEval (len 128)accuracy (%): 19.5 -> 27.4Improved
add LookUM lookahead path selectionLLaDA-8B · GSM8K (len 128)accuracy (%): 68.3 -> 72.7Improved
add LookUM to RL-tuned modelLLaDA-1.5 · GSM8K (len 256)accuracy (%): 79.4 -> 82.3Improved
add LookUMLLaDA-1.5 · Countdown (len 256)accuracy (%): 23.4 -> 17.9Hurt
add remasking (ReMDM) to confidence unmaskingLLaDA-8B · HumanEval (len 128)accuracy (%): 19.5 -> 17.1No clear effect
use average negative entropy instead of average confidence verifier (ablation)LookUM · GSM8K (len 128)accuracy (%): 71.0 -> 72.3Improved
use certainty filtering pool instead of N-best (ablation)LookUM · Countdown (len 128)accuracy (%): 32.0 -> 19.5Hurt
use NIS instead of SMC sampling (ablation)LookUM · Countdown (len 128)accuracy (%): 23.1 -> 31.3Improved
use NIS instead of SMC sampling (ablation)LookUM · MATH500 (len 128)accuracy (%): 29.0 (SMC) vs 28.6 (NIS)No clear effect
use external process reward model verifier instead of uncertainty (ablation)LookUM · MATH500 / GSM8Kaccuracy (%): 26.0 -> 24.0 (MATH500 SMC 16 particles)Hurt
increase number of paths beyond 2-4 (ablation)LookUM · GSM8K/MATH500accuracy (%)No clear effect

Experimental setup

Baselines: confidence-based sampling

Builds on: masked diffusion models

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy