Research map / Image generation / Fast diffusion sampling
Logit‑KL Flow Matching: Non‑Autoregressive Text Generation via Sampling‑Hybrid Inference
Egor Sevriugov, Nikita Dragunov, Anton Razzhigaev, Andrey Kuznetsov, Ivan Oseledets
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use KL-geodesic (logit-linear) interpolation instead of Fisher-Rao (ablation) | KL-Flow (150M) · FineFineWeb | perplexity (Llama 2): 192 -> 41 | Improved |
| use KL-geodesic instead of linear simplex interpolation (ablation) | KL-Flow (150M) · FineFineWeb | perplexity (Llama 2): 1344 -> 41 | Improved |
| scale KL-Flow from 150M to 1.5B parameters | KL-Flow · FineFineWeb, NFE 1024 | perplexity (Llama 2): 35.1 -> 32.7 | Improved |
| KL-Flow vs SEDD on unconditional generation | SEDD · FineFineWeb, NFE 1024 | perplexity (Llama 2): 47.6 -> 35.1 | Improved |
| KL-Flow vs autoregressive GPT-2 on TinyStories | GPT 2 · TinyStories | perplexity: 15.4 vs 19.0 | Hurt |
| hybrid inference vs sampling inference on Lamini Instruction (ablation) | KL-flow (sampling) · Lamini Instruction | BLEU (top-5): 7.7 -> 9.5 | Improved |
| sampling inference vs hybrid inference on WMT14 De-En (ablation) | KL-flow (hybrid) · WMT14 De-En | BLEU (top-5): 23.8 -> 27.0 | Improved |
| KL-Flow vs DFM on code infilling (10% masked) | DFM · MBPP infilling 10% | Pass@1: 11.1 -> 17.4 | Improved |
| KL-Flow vs baselines at 90% masking | GPT-2/DFM/SEDD · MBPP infilling 90% | Pass@1: 0.1 -> 0.2 | No clear effect |
| KL-Flow few-shot refinement with few steps | KL-Flow · FineFineWeb few-shot, 4 steps | entropy: 5.5 -> 3.6 | Hurt |
| KL-Flow few-shot refinement | DFM/SEDD · FineFineWeb few-shot, 16 steps | perplexity: 321.0 -> 99.8 | Improved |
Closest papers by meaning
- Flow Map Language Models: One-step Language Modeling via Continuous Denoising (NeurIPS 2026 · similarity 0.88)
- Reflection-Window Decoding: Text Generation with Selective Refinement (ICML 2025 · similarity 0.88)
- Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows (NeurIPS 2025 · similarity 0.87)
- Breaking AR’s Sampling Bottleneck: Provable Acceleration via Diffusion Language Models (NeurIPS 2025 · similarity 0.87)
- Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning (NeurIPS 2025 · similarity 0.87)
- Beyond Autoregression: Fast LLMs via Self-Distillation Through Time (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
