Research map / Image generation / Fast diffusion sampling
FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
Amin Karimi Monsefi, Nikhil Bhendawade, Manuel Rafael Ciosici, Dominic Culver, Yizhe Zhang, Irina Belousova
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cumulative scalar to RK-4 shortcut sampling (ablation) | FS-DFM 0.17B · WikiText-103 1 NFE | GPT-2 perplexity: 1\,312.65 -> 514.40 | Improved |
| add cumulative scalar to RK-4 shortcut sampling (ablation) | FS-DFM 0.17B · WikiText-103 8 NFE | GPT-2 perplexity: 97.51 -> 90.49 | Improved |
| add cumulative scalar to RK-4 shortcut sampling (ablation) | FS-DFM 0.17B · WikiText-103 1024 NFE | GPT-2 perplexity: 85.61 -> 87.36 | No clear effect |
| use RK-4 shortcut teacher instead of RK-2 (ablation) | FS-DFM · WikiText-103 across NFEs | median GPT-2 perplexity: 84 -> 80 (median); entropy 7.50 -> 7.63 | Improved |
| use uniform source instead of mask source (ablation) | FS-DFM 0.17B · WikiText-103 1 NFE | GPT-2 perplexity: 4\,438.49 -> 777.02 | Improved |
| use uniform source instead of mask source (ablation) | FS-DFM 0.17B · WikiText-103 1024 NFE | GPT-2 perplexity: 364.61 -> 93.24 | Improved |
| tail boost 20 instead of 10 on largest step size (ablation) | FS-DFM 0.17B · WikiText-103 1 NFE | GPT-2 perplexity: 834.85 -> 514.40 | Improved |
| tail boost 20 instead of 10 on largest step size (ablation) | FS-DFM 0.17B · WikiText-103 4 NFE | GPT-2 perplexity: 159.59 -> 176.19 | Hurt |
| use TB-20 instead of geometric step-size weights (ablation) | FS-DFM 0.17B · WikiText-103 1 NFE | GPT-2 perplexity: 1\,353.99 -> 514.40 | Improved |
| use TB-20 instead of annealed geometric weights (ablation) | FS-DFM 0.17B · WikiText-103 1 NFE | GPT-2 perplexity: 906.15 -> 514.40 | Improved |
| use TB-20 instead of uniform step-size weights (ablation) | FS-DFM 0.17B · WikiText-103 1 NFE | GPT-2 perplexity: 3.01 -> 514.40 | Mixed |
| fine-tune DFM with step-aware shortcut objective (FS-DFM vs DFM) | DFM 0.17B · Expanded baselines, 8 steps | perplexity: 335.64 -> 90.49 | Improved |
Later papers that compared with this method
1 later accepted paper reports FS-DFM as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation (NeurIPS 2026 · better on 7 of 8 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Breaking AR’s Sampling Bottleneck: Provable Acceleration via Diffusion Language Models (NeurIPS 2025 · similarity 0.92)
- Flow Map Language Models: One-step Language Modeling via Continuous Denoising (NeurIPS 2026 · similarity 0.91)
- Energy-Based Diffusion Language Models for Text Generation (ICLR 2025 · similarity 0.91)
- Beyond Autoregression: Fast LLMs via Self-Distillation Through Time (ICLR 2025 · similarity 0.91)
- Scaling Diffusion Language Models via Adaptation from Autoregressive Models (ICLR 2025 · similarity 0.91)
- Theoretical Benefit and Limitation of Diffusion Language Model (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
