Research map / Image generation / Fast diffusion sampling

Re-evaluating Confidence Remasking in Masked Diffusion Language Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Re-evaluates the training-free post-hoc confidence remasking method WINO against Fast-dLLM confidence unmasking for masked diffusion LLMs, finding little benefit at standard block length 32 under greedy decoding, mixed gains under sampling, and larger gains with learned Bernoulli unmaskers.

Problem

Whether training-free, post-hoc confidence-based remasking (WINO) meaningfully improves masked diffusion LLMs over strong confidence-based unmasking (Fast-dLLM), under standard and non-greedy decoding settings.

Approach

Rather than proposing a new remasking method, it carefully re-evaluates WINO against Fast-dLLM across block lengths, decoding temperatures, and unmasking strategies, measuring throughput, flip-flop rates, shadow-vs-oracle confidence, and pass@k diversity.

Key results

• At BL=32 under greedy decoding, WINO improves over Fast-dLLM's top accuracy by only ~0.4% (LLaDA) and ~0.5% (Dream) averaged across four benchmarks.
• At tau=0.8, WINO improves pass@1 by ~2.6% on average over Fast-dLLM, but the gain shrinks to ~0.7% at pass@64.
• Adding WINO remasking on top of dUltra's Bernoulli unmasker improves accuracy by ~3.2% on average at a cost of ~2 extra NFEs.

Surprising findings

• Around 75-90% (LLaDA) and 85-95% (Dream) of remasked positions are re-predicted to the same token (flip-flops), and expanding remasking to spatial or temporal neighbors degrades accuracy.
• WINO at BL=128 underperforms Fast-dLLM at BL=32, suggesting its gains mainly compensate for large-block-length degradation rather than raising the performance ceiling.

Limitations

• Evaluation restricted to WINO as representative of post-hoc confidence remasking; other methods (Saber, ReMDM, COVER) only partly examined.
• Results limited to two dLLMs (LLaDA-8B-Instruct, Dream-7B) and four math/code benchmarks.

Experimental setup

Datasets: GSM8k MATH-500 HumanEval MBPP

Benchmarks: GSM8k MATH-500 HumanEval MBPP

Metrics: accuracy NFEs throughput (tokens/second) latency pass@k flip-flop frequency

Baselines: Fast-dLLM high-confidence sampling Saber dUltra

Builds on: WINO Fast-dLLM dUltra LLaDA Dream

Closest papers by meaning

• CORE: Context-Robust Remasking for Diffusion Language Models (ICML 2026 · similarity 0.94)
• Confidence-Based Decoding is Provably Efficient for Diffusion Language Models (NeurIPS 2026 · similarity 0.92)
• Learn from Your Mistakes: Self-Correcting Masked Diffusion Models (NeurIPS 2026 · similarity 0.91)
• Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
• Remasking Discrete Diffusion Models with Inference-Time Scaling (NeurIPS 2025 · similarity 0.91)
• Simple and Effective Masked Diffusion Language Models (NeurIPS 2024 · similarity 0.91)