Research map / Probabilistic and scientific modeling / Variational and generative sampling

Distribution Matching Distillation without Fake Score Network

NeurIPS 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Score-based generative modeling analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FSF-DMD, which removes the auxiliary fake-score network from distribution matching distillation for flow-map generators by using the generator's endpoint pseudo-velocity as the fake-side signal, evaluated on ImageNet-1K 256 with LightningDiT-B/1.

Problem

DMD-style distillation of few-step generators needs an auxiliary fake-score network to track the evolving generator distribution, adding memory and update overhead and stabilization heuristics; this paper asks whether that tracker can be avoided for flow-map generators.

Approach

Under a PF-ODE coupling with injective, invertible flow maps, the fake velocity reduces to the endpoint pseudo-velocity F_theta(x_t; t, 0), so the generator itself supplies the reverse-divergence signal; this is combined with consistency distillation, a flow-map-consistent multi-step backward simulation, and a self-teacher variant for training from scratch.

Key results

• In the flow-map-initialized setting, FSF-DMD (M=2) reaches FID 3.85 versus 6.06 for consistency distillation and 4.18 for DMD2, reaching its best FID within 4K steps.
• Under flow-matching initialization, FSF-DMD achieves FID 4.70 versus 5.53 for consistency distillation and DMD2 (TTUR=1:5) and 4.90 for DMD2 + L_CD.
• In the flow-map-initialized setting, FSF-DMD uses 49GB VRAM versus 60GB for DMD2 and 0.31 vs 1.32 sec/step on B200 GPUs, per the paper's reported local setup.

Surprising findings

• FSF-DMD with single-step simulation (M=1) degrades only slightly (FID 3.96), while DMD2 with single-step simulation degrades more (4.78), which the authors attribute to the flow-map semigroup property.
• Consistency distillation with random masking alone reaches 4.98 FID, indicating a remaining gap attributed to the distributional correction.

Limitations

• Experiments are limited to ImageNet-1K 256x256 with a small B/1 backbone; scaling and text-to-image are left for future work.
• The fake-velocity identity relies on PF-ODE coupling, injectivity/invertibility and Lipschitz marginal velocity assumptions; the surrogate is a tractable replacement rather than an exact estimate.
• FSF-DMD becomes unstable when the consistency distillation loss is omitted.

Experimental setup

Datasets: ImageNet-1K

Benchmarks: ImageNet-1K 256x256

Metrics: FID50K Inception Score precision/recall

Baselines: Consistency distillation DMD2 DMD2 + IDA-style update DMD2 + L_CD TwinFlow rCM SenseFlow RCGM

Builds on: DMD DMD2 MeanFlow improved MeanFlow consistency distillation

Closest papers by meaning

• Path-Guided Flow Matching for Dataset Distillation (NeurIPS 2026 · similarity 0.91)
• Self-Corrected Flow Distillation for Consistent One-Step and Few-Step Image Generation (AAAI 2025 · similarity 0.89)
• Improved Distribution Matching Distillation for Fast Image Synthesis (NeurIPS 2024 · similarity 0.89)
• Align Your Flow: Scaling Continuous-Time Flow Map Distillation (NeurIPS 2025 · similarity 0.89)
• Drift Flow Matching (NeurIPS 2026 · similarity 0.89)
• Continuous-Time Distribution Matching for Few-Step Diffusion Distillation (NeurIPS 2026 · similarity 0.89)