Research map / Image generation / Fast diffusion sampling
Diffusion Language Models Can Approximate Optimal Infilling Lengths Implicitly
Hengchang Liu, Zhao Yang, Bing Su
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that diffusion language models implicitly reveal the correct infilling length through first-step denoising confidence patterns, and exploits this for length selection during infilling.
Problem
Diffusion language models perform infilling well only when the infilling length is pre-specified correctly, and the correct length is usually unknown in advance.
Approach
Identifies a local Oracle Peak in first-step denoising confidence near the ground-truth infilling length, indicating DLMs implicitly encode the optimal length, which can be recovered without training.
Surprising findings
- DLMs carry an implicit signal for the correct infilling length in first-step denoising confidence, without being explicitly trained for length prediction.
Closest papers by meaning
- Dynamically Structured Diffusion Language Model Decoding via Bayesian Inference (NeurIPS 2026 · similarity 0.89)
- Theoretical Benefit and Limitation of Diffusion Language Model (NeurIPS 2025 · similarity 0.88)
- Confidence-Based Decoding is Provably Efficient for Diffusion Language Models (NeurIPS 2026 · similarity 0.88)
- Denoising Time Matters: Diverse Generation in Diffusion Language Models (NeurIPS 2026 · similarity 0.88)
- PACE-dLLM: Elastic Block Decoding via Confidence Cliff Estimation for Diffusion Language Models (NeurIPS 2026 · similarity 0.88)
- Accelerating Diffusion Language Models via Structured Suffix Modeling (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
