Research map / Probabilistic and scientific modeling / Variational and generative sampling
Energy Matching: Unifying Flow Matching and Energy-Based Models for Generative Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Energy Matching, which learns a single time-independent scalar potential whose gradient drives optimal-transport-like flow far from the data and whose Boltzmann equilibrium near the data captures the likelihood, combining flow matching with EBM training.
Problem
Flow and diffusion generative models do not explicitly capture the unnormalized data likelihood, making it hard to integrate partial observations and priors, while existing EBMs have poor generation quality and are unstable to train in high dimensions.
Approach
Use a single time-independent scalar potential V(x): far from the data, its gradient defines an OT-like irrotational flow trained with a flow-matching loss; near the data manifold, a temperature-increasing Langevin regime is trained with contrastive divergence so that exp(-V/eps_max) matches the data distribution.
Key results
- Energy Matching reaches FID 3.34 on unconditional CIFAR-10 with 50M params, beating the EBM baselines (e.g., EGC 5.36, Cooperative DRL-large 3.68) and matching DDPM++ (3.45).
- On ImageNet 32x32, Energy Matching achieves FID 6.64 versus 9.35 for Cooperative DRL (40M params) among EBM methods.
- For LID estimation, Spearman correlation with PNG size is 0.877 on MNIST and 0.901 on CIFAR-10, exceeding the reported NB baseline (0.864 and 0.894).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Energy Matching instead of CLEL-large | CIFAR-10 unconditional generation · CIFAR-10 | FID: 8.61 -> 3.34 | Helped |
| Use Energy Matching instead of CLEL-large | ImageNet 32x32 unconditional generation · ImageNet 32x32 | FID: 15.47 -> 6.64 | Helped |
| Use Energy Matching instead of NCSN++ | CIFAR-10 unconditional generation · CIFAR-10 | FID: 2.45 -> 3.34 | Hurt |
| Add repulsion interaction energy (ablation) | Energy Matching · AAV medium | Diversity: 5.86 -> 6.22 | Helped |
| Add repulsion interaction energy (ablation) | Energy Matching · AAV medium | Fitness: 0.59 (0.0) -> 0.58 (0.0) | Hurt |
| Add repulsion interaction energy (ablation) | Energy Matching · AAV hard | Diversity: 4.77 -> 5.22 | Helped |
| Add repulsion interaction energy (ablation) | Energy Matching · AAV hard | Fitness: 0.61 (0.0) -> 0.60 (0.0) | Hurt |
| Replace the LP OT solver with random matching (ablation) | Energy Matching · CIFAR-10 | FID: 3.34 -> 3.37 | Hurt |
| Use Energy Matching instead of NB | LID estimation · MNIST | Spearman's correlation: 0.864 -> 0.877 | Helped |
| Use Energy Matching instead of NB | LID estimation · CIFAR-10 | Spearman's correlation: 0.894 -> 0.901 | Helped |
| Sample from a model trained with pure flow-like objective instead of combined flow-like and contrastive objectives (ablation) | Energy Matching · CIFAR-10 | FID | Hurt |
Surprising findings
- Training with a time-independent velocity field (no time conditioning) can match or outperform time-dependent flow and diffusion models in sample quality, contrary to common belief.
- Sampling-time FID for pure optimal-transport training (no contrastive phase) degrades sharply when sampling beyond time 1, indicating the EBM phase is needed for stable long-time sampling.
Limitations
- Requires an input-gradient computation, increasing GPU memory (20-40%) and per-step sampling cost.
- Full Hessian spectrum for LID estimation costs O(d^3), which is impractical for very high-dimensional data.
- Sensitive to many stabilizing hyperparameters (alpha, beta, lambda_CD, tau_s) that must be tuned empirically.
Experimental setup
Datasets: CIFAR-10 ImageNet 32x32 CelebA MNIST AAV protein dataset (medium and hard splits)
Benchmarks: CIFAR-10 ImageNet 32x32 CelebA inpainting AAV protein inverse design (medium, hard) MNIST and CIFAR-10 LID estimation
Metrics: FID Fitness Diversity (Levenshtein distance) Novelty Spearman correlation with PNG compression sampling time
Baselines: ImprovedCD CLEL Cooperative DRL EGC Hierarchical EBM Diffusion Action Matching OT-CFM Flow matching DDPM++ NCSN++ VLGPO gg-dWJS
Builds on: Flow Matching Jordan-Kinderlehrer-Otto scheme contrastive divergence OT-CFM VLGPO
Closest papers by meaning
- Equilibrium Matching: Generative Modeling with Implicit Energy-Based Models (NeurIPS 2026 · similarity 0.91)
- Energy-Based Flow Matching for Generating 3D Molecular Structure (ICML 2025 · similarity 0.89)
- BoltzNCE: Learning likelihoods for Boltzmann Generation with Stochastic Interpolants and Noise Contrastive Estimation (NeurIPS 2025 · similarity 0.89)
- Controlled Generation with Equivariant Variational Flow Matching (ICML 2025 · similarity 0.88)
- Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood (ICLR 2024 · similarity 0.88)
- Energy-based generator matching: A neural sampler for general state space (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.