Research map / Image generation / Fast diffusion sampling

Continuous-Time Distribution Matching for Few-Step Diffusion Distillation

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Fast diffusion ODE solvers · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Continuous-Time Distribution Matching (CDM), which extends DMD-style distillation from fixed discrete timesteps to continuous random-length schedules plus an off-trajectory velocity-extrapolation loss, yielding 4-step text-to-image students on SD3-Medium and Longcat-Image without GAN or reward models.

Problem

Decoupled distribution matching distillation (DMD) restricts backward simulation and supervision to a few discrete timesteps, which leaves the velocity field unsmooth and causes over-smoothing and artifacts in few-step diffusion generation, often requiring GAN or reward-model auxiliaries.

Approach

Replace the fixed discrete DMD schedule with a randomly-lengthed continuous schedule so distribution matching applies at arbitrary timesteps, and add a CDM loss that supervises off-trajectory latents obtained by a first-order Euler extrapolation along the student's own velocity, re-noised and scored by real and fake teachers.

Key results

• On SD3-Medium at 4 NFE, CDM reaches HPSv3 9.561 versus 9.176 for D-DMD (reproduced) and 8.419 for DMD2, and Aesthetic 6.075 and DPGBench 85.26 (best among compared methods).
• On Longcat-Image at 4 NFE, CDM reaches HPSv3 10.65 versus 9.629 for D-DMD and 9.450 for the 100-NFE teacher.
• Ablation on SD3-Medium shows removing the CDM loss drops HPSv3 from 9.561 to 9.153; training cost is about 1.8x D-DMD with similar peak memory (62.5 vs 62.2 GB).

Surprising findings

• The 4-NFE student matches or surpasses the 100-NFE teacher on some metrics (e.g., DPGBench and HPSv3 on both backbones).
• The DM loss alone yields students that closely match the teacher's CFG-free samples, suggesting DM is a substantive driver rather than a mere regularizer.

Limitations

• Training cost rises (about 1.8x D-DMD) due to longer average backward simulation and extra teacher forward passes on extrapolated latents.
• Distillation is upper-bounded by the teacher; concepts the teacher handles poorly are unlikely to be recovered.
• Evaluation is restricted to text-to-image generation; video and image editing are not explored.

Experimental setup

Datasets: T2I-2M LAION ShareGPT-4o-Image PickScore OCR prompts PickScore test split DPG-Bench FlowGRPO OCR prompts COCO 2014 validation

Benchmarks: DPG-Bench PickScore test split COCO 2014 FID OCR accuracy

Metrics: Aesthetic Score PickScore HPSv3 CLIPScore (ViT-H-14) DPGBench FID OCR accuracy

Baselines: Hyper-SD Flash TDM DMD2 D-DMD Longcat-Image base

Builds on: DMD DMD2 Decoupled DMD Tweedie's formula Diff-Instruct

Closest papers by meaning

• One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.91)
• Inductive Moment Matching (ICML 2025 · similarity 0.89)
• Improved Distribution Matching Distillation for Fast Image Synthesis (NeurIPS 2024 · similarity 0.89)
• CoDMD: Copula-aware Distribution Matching Distillation for Fast Video Generation (NeurIPS 2026 · similarity 0.89)
• Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch (NeurIPS 2025 · similarity 0.89)
• Distribution Matching Distillation without Fake Score Network (NeurIPS 2026 · similarity 0.89)