Research map / Image generation / Fast diffusion sampling
Inductive Moment Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Inductive Moment Matching (IMM), a single-stage, from-scratch few-step generative model that learns one-step mappings between marginals of stochastic interpolants, trained by MMD-based moment matching with bootstrapped targets.
Problem
Diffusion and flow models are slow at inference, while few-step alternatives (distillation, consistency models) suffer from training instability and require heavy tuning or pretraining.
Approach
Learn a one-step sampler mapping the marginal at time t to the marginal at any earlier time s, using self-consistent interpolants so that the model is trained by induction (matching its own distribution at an intermediate time r) with an MMD objective using Laplace kernels, which matches all moments; consistency models are shown to be a single-particle, first-moment special case.
Key results
- IMM achieves 1.99 FID on ImageNet-256 with 8 steps (XL/2, w=1.5), surpassing diffusion baselines such as DiT and SiT under the same architecture.
- IMM achieves 1.98 FID on CIFAR-10 with 2 steps for a model trained from scratch, state-of-the-art among from-scratch few-step models in the table.
- On ImageNet-256, 1-step IMM reaches 7.77 FID (XL/2, w=1.25) versus 10.60 for Shortcut models at 1 step.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase IMM sampling from 1 step to 2 steps | IMM · CIFAR-10 | FID: 3.20 -> 1.98 | Helped |
| Increase IMM sampling from 1 step to 8 steps with w=1.5 | IMM XL/2 · ImageNet-256x256 | FID: 8.05 -> 1.99 | Helped |
| Use IMM with 8 steps instead of SiT-XL/2 with w=1.5 | DiT-XL/2-based IMM · ImageNet-256x256 | FID: 2.15 -> 1.99 | Helped |
| Use constant decrement in eta_t rather than the other tested mappings (ablation) | IMM · CIFAR-10 and ImageNet-256x256 | FID | Helped |
| Use mapping-gap k=14 instead of k in {11,12,13} (ablation) | IMM · CIFAR-10 and ImageNet-256x256 | FID | Hurt |
| Use particle number M=4 rather than M=1 or M=2 (ablation) | IMM with DiT on ImageNet-256x256 · ImageNet-256x256 | FID | Helped |
| Increase particle number beyond the best-performing few-particle setting (ablation) | IMM · ImageNet-256x256 | FID | Hurt |
| Increase noise-conditioning scale c (ablation) | IMM · ImageNet-256x256 | FID | No clear effect |
| Use a=2 rather than a=1 for the weighting exponent (ablation) | IMM · Multi-step generation | FID | Helped |
| Add alpha_t weighting jointly with ELBO weighting (ablation) | IMM · CIFAR-10 and ImageNet-256x256 | FID | Helped |
| Add 1/(alpha_t^2 + sigma_t^2) weighting (ablation) | IMM · CIFAR-10 and ImageNet-256x256 | FID | Helped |
| Use Identity parameterization instead of other tested parameterizations (ablation) | IMM · CIFAR-10 and ImageNet-256x256 | FID | Hurt |
Surprising findings
- Using a very small particle number (M=2) or M=1 (consistency-model style) leads to training collapse on ImageNet with DiT, so a moderate particle count (M=4) is needed.
Limitations
- Results are reported only on CIFAR-10 and class-conditional ImageNet-256; no text-to-image or video experiments.
- Continuous-time extension is derived but left to future work.
- Multi-step pushforward with restart samplers is not explored.
Experimental setup
Datasets: CIFAR-10 ImageNet-256
Benchmarks: CIFAR-10 ImageNet-256
Metrics: FID
Baselines: consistency models (iCT) Shortcut models EDM DiT-XL/2 SiT-XL/2 VAR CTM SiD MAR
Builds on: stochastic interpolants consistency models EDM DDIM DiT
Closest papers by meaning
- Multistep Distillation of Diffusion Models via Moment Matching (NeurIPS 2024 · similarity 0.94)
- Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch (NeurIPS 2025 · similarity 0.91)
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.90)
- One-Step Diffusion Distillation through Score Implicit Matching (NeurIPS 2024 · similarity 0.90)
- Continuous-Time Distribution Matching for Few-Step Diffusion Distillation (NeurIPS 2026 · similarity 0.89)
- Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.