Research map / Learning theory and algorithms / Optimal transport
Generative Sliced MMD Flows with Riesz Kernels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that MMD with negative-distance (Riesz r=1) kernels equals its sliced version up to a constant, enabling O((M+N)log(M+N)) sorting-based gradients, and uses these sliced MMD flows to train iterative UNet-based generative models on MNIST, FashionMNIST, CIFAR10 and CelebA.
Problem
MMD gradient flows are computationally expensive because MMD between empirical measures costs quadratically in the number of support points and the approach has been limited to small particle sets and low dimensions.
Approach
MMD with Riesz kernels (energy distance) coincides with the sliced MMD of univariate Riesz kernels up to a dimension-dependent constant, so gradients reduce to 1D computations that can be done by sorting; slices are then approximated with P random projections, and a sequence of UNets approximates the particle flow.
Key results
- Generative sliced MMD flow achieves FID 3.1 on MNIST, 11.3 on FashionMNIST, 54.8 on CIFAR10 and 32.1 on CelebA, outperforming the listed flow-based baselines such as SIG (4.5, 13.7, 66.5, 37.3).
- Relative error of the sliced MMD gradient behaves as O(sqrt(d/P)) in numerical experiments over dimension d and projection count P.
- For r=1 the 1D gradient is computed by sorting with complexity O((M+N)log(M+N)) instead of O(MN+N^2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Generative Sliced MMD Flow instead of MMD GAN | Image generation · MNIST | FID: 4.2 -> 3.1 ± 0.06 | Helped |
| Use Generative Sliced MMD Flow instead of SIG | Image generation · FashionMNIST | FID: 13.7 -> 11.3 ± 0.07 | Helped |
| Use Generative Sliced MMD Flow instead of NCSN | Image generation · CIFAR10 | FID: 25.3 -> 54.8 ± 0.26 | Hurt |
| Use Generative Sliced MMD Flow instead of MMD GAN | Image generation · CelebA | FID: 29.2 -> 32.1 ± 0.17 | Hurt |
| Add momentum to the MMD flow (ablation) | MMD particle flow · MNIST and CIFAR10 | convergence speed | Helped |
| Use only P=10 or P=100 projections (ablation) | Generative Sliced MMD Flow · MNIST | FID | Hurt |
| Increase projections from P=500 to P=1000 (ablation) | Generative Sliced MMD Flow · MNIST | FID | Helped |
Surprising findings
- The MMD with Riesz kernel is equivalent to its sliced version, so the Riesz kernel gives a Wasserstein-1 based lower bound on MMD for compactly supported measures, and the stated conjecture from prior work only holds for non-compactly supported measures.
Limitations
- Batching is not easily possible for interacting particle methods, since the flow for one training set does not transfer to another.
- Taking full projections scales with data dimension d; local projections could be more efficient.
- Error bound scales with sqrt(d), requiring O(d) projections to reach a fixed error, so complexity is quadratic in dimension.
Experimental setup
Datasets: MNIST FashionMNIST CIFAR10 CelebA
Benchmarks: MNIST FashionMNIST CIFAR10 CelebA
Metrics: FID relative gradient approximation error run time
Baselines: CWAE SWF+Autoencoder+RealNVP 2-stage VAE GLF WGAN MMD GAN NCSN SWF SIG l-SWF
Builds on: MMD gradient flows sliced Wasserstein distance Wasserstein gradient flows UNet
Closest papers by meaning
- Sobolev Regularized MMD Gradient Flow (NeurIPS 2026 · similarity 0.90)
- Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernel (ICLR 2024 · similarity 0.87)
- Deep MMD Gradient Flow without adversarial training (ICLR 2025 · similarity 0.86)
- Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel (NeurIPS 2025 · similarity 0.84)
- Differentiable Generalized Sliced Wasserstein Plans (NeurIPS 2025 · similarity 0.84)
- Simplicity Bias via Global Convergence of Sharpness Minimization (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.