AI Paper Atlas

Research map / Image generation / Fast diffusion sampling

Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from $k$-Parity

Jianhao Huang, Baharan Mirzasoleiman

ICML 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
restrict mask-rate sampling to t in [0.45, 0.55] during pretrainingLLaDA-8B pretraining from scratch · HellaSwagaccuracy: 0.354 -> 0.400Improved
restrict mask-rate sampling to t in [0.45, 0.55] during pretrainingLLaDA-8B pretraining from scratch · ARC-Easyaccuracy: 0.342 -> 0.430Improved
restrict mask-rate sampling to t in [0.45, 0.55] in SFTLLaDA-8B Base SFT · GPQAaccuracy: 0.344 -> 0.402Improved
restrict mask-rate sampling to t in [0.45, 0.55] in SFTLLaDA-8B Base SFT · MMLU, MMLU-stem, ARC-Challengeaccuracy: MMLU 0.659 -> 0.669; ARC-Challenge 0.468 -> 0.480Improved
restrict mask-rate sampling to the mid-range window [0.45, 0.55] in SFTLLaDA-8B Base SFT · GSM8Kaccuracy: 0.768 -> 0.738Hurt
restrict mask-rate sampling to the mid-range window [0.45, 0.55] in SFTLLaDA-8B Base SFT · MATHaccuracy: 0.341 -> 0.342No clear effect
shift sampling interval toward t=1, window [0.5, 1.0] instead of [0, 1] (ablation)LLaDA-8B Base SFT · GSM8Kaccuracy: 0.768 -> 0.785Improved
shift sampling interval toward t=1, window [0.5, 1.0] instead of [0, 1] (ablation)LLaDA-8B Base SFT · MATHaccuracy: 0.341 -> 0.375Improved
train only on mask rates near the boundaries (t in [0, 0.1] or [0.9, 1.0]) (ablation)50M nanoGPT MDLM · WikiText test losstest lossHurt
train only on mid-range mask rates t in [0.4, 0.5] or [0.5, 0.6] (ablation)50M nanoGPT MDLM · WikiText test losstest loss: 3.88 -> 3.62Improved
use masked diffusion objective instead of direct output supervisionnanoGPT on (20,6) parity · k-parityvalidation accuracy / grokkingImproved
mask-rate range U[0,0.2] or U[0,0.3] rather than U[0,0.1] or U[0,0.4] (ablation)nanoGPT MD on (20,6) parity · k-parityconvergence speedImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy