Research map / Image generation / Fast diffusion sampling

ELF: Embedded Language Flows

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ELF, a continuous-time flow matching diffusion language model that denoises in frozen T5 embedding space with a shared-weight final decoding step, adding training-time CFG and self-conditioning.

Problem

Continuous diffusion language models lag discrete DLMs; it is unclear whether the gap stems from the discrete nature of language or from algorithmic design choices, and existing continuous DLMs use per-step token-level losses or separate decoders.

Approach

Perform flow matching entirely in a frozen pretrained contextual embedding space (x-prediction, rectified flow), discretizing only at the final time step with a shared-weight network trained with cross-entropy, so no separate decoder is needed and image-style CFG can be applied directly via training-time CFG with self-conditioning.

Key results

• ELF-B reaches generative perplexity 24 on OWT with 32 sampling steps (SDE sampler, self-conditioning CFG 3), with 105M parameters versus ~170M for baselines.
• ELF-B achieves 26.4 BLEU on WMT14 De-En and 36.0 ROUGE-1 on XSum, best among compared methods of similar scale.
• ELF uses about 45B training tokens versus over 500B for prior DLMs; ELF+PD distillation reaches 21.32 Gen. PPL at 32 steps and 136.10 at 1 step.

Surprising findings

• v-prediction works poorly when weights are shared with the final discretization step, so x-prediction is needed.
• Shared-weight denoiser-decoder matches a separately trained two-stage decoder and extends further toward low generative perplexity.

Limitations

• Performance degrades with very few sampling steps (1-4 steps) without distillation.
• Evaluation uses generative perplexity under GPT-2 Large, which is an imperfect quality proxy; no validation perplexity reported for likelihood.

Experimental setup

Datasets: OpenWebText WMT14 German-English XSum

Benchmarks: OpenWebText WMT14 De-En XSum

Metrics: generative perplexity unigram entropy BLEU ROUGE-1 ROUGE-2 ROUGE-L

Baselines: MDLM Duo FLM LangFlow E2D2 CDCD SeqDiffuSeq MDLM+SDTT Duo+DCD FMLM AR

Builds on: Flow Matching rectified flow classifier-free guidance self-conditioning Muon optimizer progressive distillation

Closest papers by meaning

• LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling (NeurIPS 2026 · similarity 0.92)
• Flow Map Language Models: One-step Language Modeling via Continuous Denoising (NeurIPS 2026 · similarity 0.90)
• Simple and Effective Masked Diffusion Language Models (NeurIPS 2024 · similarity 0.88)
• Continuous Latent Diffusion Language Model (NeurIPS 2026 · similarity 0.87)
• Rare Text Semantics Were Always There in Your Diffusion Transformer (NeurIPS 2025 · similarity 0.87)
• Discrete Modeling via Boundary Conditional Diffusion Processes (NeurIPS 2024 · similarity 0.87)