Research map / Probabilistic and scientific modeling / Variational and generative sampling
Deep MMD Gradient Flow without adversarial training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a noise-conditional MMD discriminator on diffusion-noised data without adversarial training and samples by noise-adaptive MMD gradient flow (DMMD), with a linear-kernel approximation for single-particle sampling.
Problem
Wasserstein gradient flows built on GAN-style discriminators (discriminator flows) need adversarial training of a sequence of discriminators and their image generation quality lags behind diffusion models and GANs; fixed-kernel MMD flows do not scale to images.
Approach
Train a single noise-conditional MMD discriminator on clean data versus forward-diffused noisy data across noise levels (no min-max game), then transport particles from Gaussian noise by a noise-adaptive MMD gradient flow that decreases noise level step by step; a linear base kernel allows an O(N) approximation that samples particles independently.
Key results
- On CIFAR-10, DMMD reaches FID 8.31 (Inception Score 9.09) with 100 NFE, improving over the MMD GAN implementation (FID 13.62) and discriminator flow baselines such as JKO-Flow (FID 23.10) and DGGF-KL (FID 28.80).
- DDPM (orig., FID 3.17 at 1000 NFE; impl., FID 5.19 at 100 NFE) still outperforms DMMD on CIFAR-10.
- FID on other datasets with 100 NFE: MNIST 3.0, CelebA 8.3, LSUN Church 6.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use noise-adaptive DMMD instead of MMD-GAN (impl.) | CIFAR-10 image generation · CIFAR-10 | FID: 13.62 -> 8.31 | Helped |
| use noise-adaptive DMMD instead of GS-MMD-RK with a fixed kernel | CIFAR-10 image generation · CIFAR-10 | FID: 55.00 -> 8.31 | Helped |
| use DMMD instead of DDPM (orig.) | CIFAR-10 image generation · CIFAR-10 | FID: 3.17 -> 8.31 | Hurt |
| use DMMD instead of discriminator-flow baselines | CIFAR-10 image generation · CIFAR-10 | FID: 28.80 -> 8.31; 23.10 -> 8.31 | Helped |
| replace exact particle interaction with approximate sampling (ablation) | DMMD · CIFAR-10 | FID: 8.31 -> 24.86 | Hurt |
| add final denoising steps to approximate sampling (ablation) | a-DMMD · CIFAR-10 | FID: 24.86 -> 9.185 | Helped |
| use approximate rather than exact denoising steps after approximate sampling (ablation) | a-DMMD · CIFAR-10 | FID: 9.185 -> 11.22 | Hurt |
| add final denoising steps (ablation) | DMMD · CIFAR-10 | FID: 8.31 -> 8.21 | Helped |
| add final denoising steps (ablation) | DMMD · CIFAR-10 | Inception Score: 9.09 -> 8.99 | Hurt |
Surprising findings
- The MMD-GAN discriminator used directly as a gradient flow (MMD-GAN-Flow) fails completely (FID 450), since its gradients vanish far from the target distribution.
- A single-particle approximate sampler using precomputed average features (a-DMMD) gets FID 24.86 before denoising, showing the interaction between particles is only partly needed.
Limitations
- Still behind DDPM on CIFAR-10 in FID, and the U-Net and noise schedule were tuned for DDPM.
- Approximate single-particle sampler with linear kernel degrades FID notably without a denoising step.
- No convergence guarantees for the flow in general settings; theory is limited to a Gaussian toy analysis.
Experimental setup
Datasets: CIFAR-10 MNIST CelebA LSUN Church
Benchmarks: CIFAR-10 MNIST CelebA LSUN Church
Metrics: FID Inception Score
Baselines: MMD GAN DDPM DGGF-KL JKO-Flow GS-MMD-RK MMD-GAN-Flow D-KALE-flow
Builds on: MMD GAN MMD gradient flow DDPM discriminator flows (DGGF) KALE flow
Closest papers by meaning
- Sobolev Regularized MMD Gradient Flow (NeurIPS 2026 · similarity 0.89)
- Bridging Data Gaps in Diffusion Models with Adversarial Noise-Based Transfer Learning (ICML 2024 · similarity 0.88)
- Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding (ICML 2024 · similarity 0.87)
- Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation (ICML 2025 · similarity 0.87)
- Structure-Guided Adversarial Training of Diffusion Models (CVPR 2024 · similarity 0.87)
- Generalization in diffusion models arises from geometry-adaptive harmonic representations (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.