Research map / Image generation / Fast diffusion sampling
Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models
Zemin Huang, Yuhang Wang, Zhiyang Chen, Guo-Jun Qi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Remask SFT (train on incorrect-token noise with remask labels) instead of vanilla SFT (ablation) | RemeDi · MATH-500 | accuracy: 40.1 -> 42.7 | Improved |
| add Remask SFT instead of vanilla SFT (ablation) | RemeDi · HumanEval | pass@1: 48.2 -> 50.0 | Improved |
| add Remask SFT instead of vanilla SFT (ablation) | RemeDi · GSM8K and MBPP | accuracy: GSM8K 83.1 -> 83.6; MBPP 43.4 -> 44.0 | Improved |
| add Remask RL after Remask SFT (ablation) | RemeDi · AlpacaEval | win rate: 12.5 -> 24.8 | Improved |
| add Remask RL after Remask SFT (ablation) | RemeDi · GSM8K, MATH, HumanEval | accuracy: GSM8K 86.3 -> 89.1; MATH 51.4 -> 52.9; HumanEval 71.3 -> 73.2 | Improved |
| add Remask RL after Remask SFT (ablation) | RemeDi · GPQA | accuracy: 32.6 -> 29.5 | Hurt |
| use Remask RL instead of LLaDOU RL on LLaDA (ablation) | GSM8K RL training · GSM8K pass@1, 200 steps | pass@1: 82.35% -> 83.33% at step 200; 77.58% -> 80.00% at step 50 | Improved |
| add remasking-enabled training on LLaDA (RemeDi vs LLaDA) | LLaDA-8B-Instruct · HumanEval | pass@1: 45.7 -> 73.2 | Improved |
| add inference-only random remasking (ReMDM) to LLaDA | LLaDA · MATH | accuracy: 38.9 -> 38.5 | No clear effect |
| harder problems on MATH-500 (level 1 to level 5) | RemeDi · MATH-500 | remasking frequency per block: 9.13 -> 13.95 tokens per block | Improved |
Closest papers by meaning
- Re-evaluating Confidence Remasking in Masked Diffusion Language Models (NeurIPS 2026 · similarity 0.93)
- CORE: Context-Robust Remasking for Diffusion Language Models (ICML 2026 · similarity 0.92)
- Remasking Discrete Diffusion Models with Inference-Time Scaling (NeurIPS 2025 · similarity 0.91)
- Diffusion-State Policy Optimization for Masked Diffusion Language Models (NeurIPS 2026 · similarity 0.90)
- Scaling up Masked Diffusion Models on Text (ICLR 2025 · similarity 0.90)
- Theoretical Benefit and Limitation of Diffusion Language Model (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
