Research map / Image generation / Fast diffusion sampling

Uncertainty Quantification for Large Language Diffusion Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies zero-shot uncertainty quantification for masked large language diffusion models, proposing diffusion-specific signals (trajectory dissimilarity, MCNLL, NFE, remasking) and D-CoCoA, a diffusion-aware adaptation of CoCoA that approaches sampling-based baselines at much lower cost.

Problem

Hallucination detection and uncertainty quantification for large language diffusion models (LLDMs) is misaligned with existing UQ methods, which assume autoregressive likelihoods or rely on costly repeated sampling that negates LLDM efficiency.

Approach

Uses intermediate denoising-trajectory states instead of repeated samples: the average dissimilarity between each intermediate argmax prediction and the final output is proposed as an uncertainty score and theoretically linked (as a lower bound) to the masked diffusion loss; it is combined with MC masked NLL and an inference term in a D-CoCoA adaptation of CoCoA.

Key results

• D-CoCoA-L requires roughly 100x less additional computation time than the best sampling-based methods (CoCoA-MCNLL and KLE) while giving only modest gains for those methods (e.g., 2.4% for LLaDA).
• On LLaDA-7B-v1.5, the best diffusion-specific D-CoCoA variants approach the PRR of expensive sampling-based baselines across eight datasets.
• Among autoregressive-designed baselines, black-box KLE is the strongest on LLaDA, while supervised feature-based methods (SATRMD, LookBackLens) perform poorly.

Surprising findings

• Information-based methods based on logits or attention maps perform only marginally better than random on LLDMs.
• Supervised hallucination detectors (SATRMD, LookBackLens) generally underperform unsupervised black-box sampling methods on LLDMs.

Limitations

• Results are reported for only two 7B LLDMs and selected tasks, limiting generality.
• Proposed D-CoCoA does not clearly outperform the best sampling-based baselines; it approaches them.
• The theoretical bound relies on strong assumptions such as perfect calibration (Fisher consistency).

Experimental setup

Datasets: MMLU TriviaQA CoQA GSM8k SamSum XSum WMT14 fr-en WMT19 de-en

Benchmarks: MMLU TriviaQA CoQA GSM8k SamSum XSum WMT14 fr-en WMT19 de-en

Metrics: PRR ROC-AUC

Baselines: MSP Perplexity AttScore CCP RAUQ MCNSE Semantic Entropy SAR EigenScore CoCoA-MTE Semantic Density LUQ

Builds on: CoCoA Semantic Entropy LLaDA Monte Carlo sequence entropy

Closest papers by meaning

• FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion (ICLR 2026 · similarity 0.91)
• Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning (NeurIPS 2025 · similarity 0.90)
• Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models (ICLR 2025 · similarity 0.89)
• Simple and Effective Masked Diffusion Language Models (NeurIPS 2024 · similarity 0.89)
• Denoising Time Matters: Diverse Generation in Diffusion Language Models (NeurIPS 2026 · similarity 0.89)
• UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Models (ICLR 2026 · similarity 0.89)